-
Two-Dimensional Unknown View Tomography from Unknown Angle Distributions
Authors:
Kaishva Chintan Shah,
Karthik S. Gurumoorthy,
Ajit Rajwade
Abstract:
This study presents a technique for 2D tomography under unknown viewing angles when the distribution of the viewing angles is also unknown. Unknown view tomography (UVT) is a problem encountered in cryo-electron microscopy and in the geometric calibration of CT systems. There exists a moderate-sized literature on the 2D UVT problem, but most existing 2D UVT algorithms assume knowledge of the angle…
▽ More
This study presents a technique for 2D tomography under unknown viewing angles when the distribution of the viewing angles is also unknown. Unknown view tomography (UVT) is a problem encountered in cryo-electron microscopy and in the geometric calibration of CT systems. There exists a moderate-sized literature on the 2D UVT problem, but most existing 2D UVT algorithms assume knowledge of the angle distribution which is not available usually. Our proposed methodology formulates the problem as an optimization task based on cross-validation error, to estimate the angle distribution jointly with the underlying 2D structure in an alternating fashion. We explore the algorithm's capabilities for the case of two probability distribution models: a semi-parametric mixture of von Mises densities and a probability mass function model. We evaluate our algorithm's performance under noisy projections using a PCA-based denoising technique and Graph Laplacian Tomography (GLT) driven by order statistics of the estimated distribution, to ensure near-perfect ordering, and compare our algorithm to intuitive baselines.
△ Less
Submitted 6 January, 2025;
originally announced January 2025.
-
Submodular Framework for Structured-Sparse Optimal Transport
Authors:
Piyushi Manupriya,
Pratik Jawanpuria,
Karthik S. Gurumoorthy,
SakethaNath Jagarlapudi,
Bamdev Mishra
Abstract:
Unbalanced optimal transport (UOT) has recently gained much attention due to its flexible framework for handling un-normalized measures and its robustness properties. In this work, we explore learning (structured) sparse transport plans in the UOT setting, i.e., transport plans have an upper bound on the number of non-sparse entries in each column (structured sparse pattern) or in the whole plan (…
▽ More
Unbalanced optimal transport (UOT) has recently gained much attention due to its flexible framework for handling un-normalized measures and its robustness properties. In this work, we explore learning (structured) sparse transport plans in the UOT setting, i.e., transport plans have an upper bound on the number of non-sparse entries in each column (structured sparse pattern) or in the whole plan (general sparse pattern). We propose novel sparsity-constrained UOT formulations building on the recently explored maximum mean discrepancy based UOT. We show that the proposed optimization problem is equivalent to the maximization of a weakly submodular function over a uniform matroid or a partition matroid. We develop efficient gradient-based discrete greedy algorithms and provide the corresponding theoretical guarantees. Empirically, we observe that our proposed greedy algorithms select a diverse support set and we illustrate the efficacy of the proposed approach in various applications.
△ Less
Submitted 31 January, 2025; v1 submitted 7 June, 2024;
originally announced June 2024.
-
INVALS: A Forward Looking Inventory Allocation System
Authors:
Shiv Krishna Jaiswal,
Karthik S. Gurumoorthy,
Etika Agarwal,
Shantala Manchenahally
Abstract:
We design an Inventory Allocation System (INVALS) that, for each item-store combination, plans the quantity to be allocated from a warehouse that replenishes multiple stores using trailers, while respecting typical operational constraints. We formulate a linear objective function which, when maximized, determines the allocation plan by considering not only the immediate store needs, but also its f…
▽ More
We design an Inventory Allocation System (INVALS) that, for each item-store combination, plans the quantity to be allocated from a warehouse that replenishes multiple stores using trailers, while respecting typical operational constraints. We formulate a linear objective function which, when maximized, determines the allocation plan by considering not only the immediate store needs, but also its future (forward) expected demand. This forward-looking allocation significantly improves the utilization of labor and trailers in the warehouse. To reduce overstocking, we adapt from our objective to prioritize allocating those items in excess which are sold faster at the stores, keeping the days of supply (DOS) to a minimum. For the proposed formulation, which is an instance of Mixed Integer Linear Programming (MILP), we present a scalable algorithm using the concepts of submodularity and optimal transport theory by: (i) sequentially adding trailers to stores based on maximum incremental gain, (ii) transforming the resultant linear program (LP) instance to an instance of capacity constrained optimal transport (COT), solvable using double entropic regularization and incurring the same computational complexity as the Sinkhorn algorithm. Compared against the planning engine that only allocates for immediate store needs, INVALS increases labor utilization by 34.70% and item occupancy in trailers by 37.08% on average. The DOS distribution is also skewed to the left, indicating that higher-demand items are allocated in excess, reducing the days they are stocked. We empirically observed that for ~90% of the replenishment cycles, the allocation results of INVALS are identical to the globally optimal MILP solution.
△ Less
Submitted 11 March, 2025; v1 submitted 6 October, 2023;
originally announced October 2023.
-
A scalable solution for the extended multi-channel facility location problem
Authors:
Etika Agarwal,
Karthik S. Gurumoorthy,
Ankit Ajit Jain,
Shantala Manchenahally
Abstract:
We study the extended version of the non-uniform, capacitated facility location problem with multiple fulfilment channels between the facilities and clients, each with their own channel capacities and service cost. Though the problem has been extensively studied in the literature, all the prior works assume a single channel of fulfilment, and the existing methods based on linear programming, prima…
▽ More
We study the extended version of the non-uniform, capacitated facility location problem with multiple fulfilment channels between the facilities and clients, each with their own channel capacities and service cost. Though the problem has been extensively studied in the literature, all the prior works assume a single channel of fulfilment, and the existing methods based on linear programming, primal-dual relationships, local search heuristics etc. do not scale for a large supply chain system involving millions of decision variables. Using the concepts of sub-modularity and optimal transport theory, we present a scalable algorithm for determining the set of facilities to be opened under a cardinality constraint. By introducing various schemes such as: (i) iterative facility selection using incremental gain, (ii) approximation of the linear program using novel multi-stage Sinkhorn iterations, (iii) creation of facilities one for each fulfilment channel etc., we develop a fast but a tight approximate solution, requiring $\mathcal{O}\left(\frac{3+k}{m}ln\left(\frac{1}ε\right)\right)$ instances of optimal transport problems to select k facilities from m options, each solvable in linear time. Our algorithm is implicitly endowed with all the theoretical guarantees enjoyed by submodular maximisation problems and the Sinkhorn distances. When compared against the state-of-the-art commercial MILP solvers, we obtain a 100-fold speedup in computation, while the difference in objective values lies within a narrow range of 3%.
△ Less
Submitted 21 April, 2023;
originally announced April 2023.
-
Cooperative Multi-Agent Reinforcement Learning for Inventory Management
Authors:
Madhav Khirwar,
Karthik S. Gurumoorthy,
Ankit Ajit Jain,
Shantala Manchenahally
Abstract:
With Reinforcement Learning (RL) for inventory management (IM) being a nascent field of research, approaches tend to be limited to simple, linear environments with implementations that are minor modifications of off-the-shelf RL algorithms. Scaling these simplistic environments to a real-world supply chain comes with a few challenges such as: minimizing the computational requirements of the enviro…
▽ More
With Reinforcement Learning (RL) for inventory management (IM) being a nascent field of research, approaches tend to be limited to simple, linear environments with implementations that are minor modifications of off-the-shelf RL algorithms. Scaling these simplistic environments to a real-world supply chain comes with a few challenges such as: minimizing the computational requirements of the environment, specifying agent configurations that are representative of dynamics at real world stores and warehouses, and specifying a reward framework that encourages desirable behavior across the whole supply chain. In this work, we present a system with a custom GPU-parallelized environment that consists of one warehouse and multiple stores, a novel architecture for agent-environment dynamics incorporating enhanced state and action spaces, and a shared reward specification that seeks to optimize for a large retailer's supply chain needs. Each vertex in the supply chain graph is an independent agent that, based on its own inventory, able to place replenishment orders to the vertex upstream. The warehouse agent, aside from placing orders from the supplier, has the special property of also being able to constrain replenishment to stores downstream, which results in it learning an additional allocation sub-policy. We achieve a system that outperforms standard inventory control policies such as a base-stock policy and other RL-based specifications for 1 product, and lay out a future direction of work for multiple products.
△ Less
Submitted 18 April, 2023;
originally announced April 2023.
-
Estimating Joint Probability Distribution With Low-Rank Tensor Decomposition, Radon Transforms and Dictionaries
Authors:
Pranava Singhal,
Waqar Mirza,
Ajit Rajwade,
Karthik S. Gurumoorthy
Abstract:
In this paper, we describe a method for estimating the joint probability density from data samples by assuming that the underlying distribution can be decomposed as a mixture of product densities with few mixture components. Prior works have used such a decomposition to estimate the joint density from lower-dimensional marginals, which can be estimated more reliably with the same number of samples…
▽ More
In this paper, we describe a method for estimating the joint probability density from data samples by assuming that the underlying distribution can be decomposed as a mixture of product densities with few mixture components. Prior works have used such a decomposition to estimate the joint density from lower-dimensional marginals, which can be estimated more reliably with the same number of samples. We combine two key ideas: dictionaries to represent 1-D densities, and random projections to estimate the joint distribution from 1-D marginals, explored separately in prior work. Our algorithm benefits from improved sample complexity over the previous dictionary-based approach by using 1-D marginals for reconstruction. We evaluate the performance of our method on estimating synthetic probability densities and compare it with the previous dictionary-based approach and Gaussian Mixture Models (GMMs). Our algorithm outperforms these other approaches in all the experimental settings.
△ Less
Submitted 18 April, 2023;
originally announced April 2023.
-
Signal Reconstruction from Samples at Unknown Locations with Application to 2D Unknown View Tomography
Authors:
Sheel Shah,
Kaishva Shah,
Karthik S. Gurumoorthy,
Ajit Rajwade
Abstract:
It is well known that a band-limited signal can be reconstructed from its uniformly spaced samples if the sampling rate is sufficiently high. More recently, it has been proved that one can reconstruct a 1D band-limited signal even if the exact sample locations are unknown, but given a uniform distribution of the sample locations and their ordering in 1D. In this work, we extend the analytical erro…
▽ More
It is well known that a band-limited signal can be reconstructed from its uniformly spaced samples if the sampling rate is sufficiently high. More recently, it has been proved that one can reconstruct a 1D band-limited signal even if the exact sample locations are unknown, but given a uniform distribution of the sample locations and their ordering in 1D. In this work, we extend the analytical error bounds in such scenarios for quasi-bandlimited (QBL) signals, and for the case of arbitrary but known sampling distributions. We also prove that such reconstruction methods are resilient to a certain proportion of errors in the specification of the sample location ordering. We then express the problem of tomographic reconstruction of 2D images from 1D Radon projections under unknown angles (2D UVT) with known angle distribution, as a special case for reconstruction of QBL signals from samples at unknown locations with known distribution. Building upon our theoretical background, we present asymptotic bounds for 2D QBL image reconstruction from 1D Radon projections in the unknown angles setting, and present an extensive set of simulations to verify these bounds in varied parameter regimes. To the best of our knowledge, this is the first piece of work to perform such an analysis for 2D UVT and explicitly relate it to advances in sampling theory, even though the associated reconstruction algorithms have been known for a long time.
△ Less
Submitted 18 December, 2024; v1 submitted 13 April, 2023;
originally announced April 2023.
-
Joint Probability Estimation Using Tensor Decomposition and Dictionaries
Authors:
Shaan ul Haque,
Ajit Rajwade,
Karthik S. Gurumoorthy
Abstract:
In this work, we study non-parametric estimation of joint probabilities of a given set of discrete and continuous random variables from their (empirically estimated) 2D marginals, under the assumption that the joint probability could be decomposed and approximated by a mixture of product densities/mass functions. The problem of estimating the joint probability density function (PDF) using semi-par…
▽ More
In this work, we study non-parametric estimation of joint probabilities of a given set of discrete and continuous random variables from their (empirically estimated) 2D marginals, under the assumption that the joint probability could be decomposed and approximated by a mixture of product densities/mass functions. The problem of estimating the joint probability density function (PDF) using semi-parametric techniques such as Gaussian Mixture Models (GMMs) is widely studied. However such techniques yield poor results when the underlying densities are mixtures of various other families of distributions such as Laplacian or generalized Gaussian, uniform, Cauchy, etc. Further, GMMs are not the best choice to estimate joint distributions which are hybrid in nature, i.e., some random variables are discrete while others are continuous. We present a novel approach for estimating the PDF using ideas from dictionary representations in signal processing coupled with low rank tensor decompositions. To the best our knowledge, this is the first work on estimating joint PDFs employing dictionaries alongside tensor decompositions. We create a dictionary of various families of distributions by inspecting the data, and use it to approximate each decomposed factor of the product in the mixture. Our approach can naturally handle hybrid $N$-dimensional distributions. We test our approach on a variety of synthetic and real datasets to demonstrate its effectiveness in terms of better classification rates and lower error rates, when compared to state of the art estimators.
△ Less
Submitted 3 March, 2022;
originally announced March 2022.
-
A decision-tree framework to select optimal box-sizes for product shipments
Authors:
Karthik S. Gurumoorthy,
Abhiraj Hinge
Abstract:
In package-handling facilities, boxes of varying sizes are used to ship products. Improperly sized boxes with box dimensions much larger than the product dimensions create wastage and unduly increase the shipping costs. Since it is infeasible to make unique, tailor-made boxes for each of the $N$ products, the fundamental question that confronts e-commerce companies is: How many $K << N$ cuboidal b…
▽ More
In package-handling facilities, boxes of varying sizes are used to ship products. Improperly sized boxes with box dimensions much larger than the product dimensions create wastage and unduly increase the shipping costs. Since it is infeasible to make unique, tailor-made boxes for each of the $N$ products, the fundamental question that confronts e-commerce companies is: How many $K << N$ cuboidal boxes need to manufactured and what should be their dimensions? In this paper, we propose a solution for the single-count shipment containing one product per box in two steps: (i) reduce it to a clustering problem in the $3$ dimensional space of length, width and height where each cluster corresponds to the group of products that will be shipped in a particular size variant, and (ii) present an efficient forward-backward decision tree based clustering method with low computational complexity on $N$ and $K$ to obtain these $K$ clusters and corresponding box dimensions. Our algorithm has multiple constituent parts, each specifically designed to achieve a high-quality clustering solution. As our method generates clusters in an incremental fashion without discarding the present solution, adding or deleting a size variant is as simple as stopping the backward pass early or executing it for one more iteration. We tested the efficacy of our approach by simulating actual single-count shipments that were transported during a month by Amazon using the proposed box dimensions. Even by just modifying the existing box dimensions and not adding a new size variant, we achieved a reduction of $4.4\%$ in the shipment volume, contributing to the decrease in non-utilized, air volume space by $2.2\%$. The reduction in shipment volume and air volume improved significantly to $10.3\%$ and $6.1\%$ when we introduced $4$ additional boxes.
△ Less
Submitted 8 February, 2022;
originally announced February 2022.
-
Individual Treatment Effect Estimation Through Controlled Neural Network Training in Two Stages
Authors:
Naveen Nair,
Karthik S. Gurumoorthy,
Dinesh Mandalapu
Abstract:
We develop a Causal-Deep Neural Network (CDNN) model trained in two stages to infer causal impact estimates at an individual unit level. Using only the pre-treatment features in stage 1 in the absence of any treatment information, we learn an encoding for the covariates that best represents the outcome. In the $2^{nd}$ stage we further seek to predict the unexplained outcome from stage 1, by intro…
▽ More
We develop a Causal-Deep Neural Network (CDNN) model trained in two stages to infer causal impact estimates at an individual unit level. Using only the pre-treatment features in stage 1 in the absence of any treatment information, we learn an encoding for the covariates that best represents the outcome. In the $2^{nd}$ stage we further seek to predict the unexplained outcome from stage 1, by introducing the treatment indicator variables alongside the encoded covariates. We prove that even without explicitly computing the treatment residual, our method still satisfies the desirable local Neyman orthogonality, making it robust to small perturbations in the nuisance parameters. Furthermore, by establishing connections with the representation learning approaches, we create a framework from which multiple variants of our algorithm can be derived. We perform initial experiments on the publicly available data sets to compare these variants and get guidance in selecting the best variant of our CDNN method. On evaluating CDNN against the state-of-the-art approaches on three benchmarking datasets, we observe that CDNN is highly competitive and often yields the most accurate individual treatment effect estimates. We highlight the strong merits of CDNN in terms of its extensibility to multiple use cases.
△ Less
Submitted 21 January, 2022;
originally announced January 2022.
-
Recovery of Joint Probability Distribution from one-way marginals: Low rank Tensors and Random Projections
Authors:
Jian Vora,
Karthik S. Gurumoorthy,
Ajit Rajwade
Abstract:
Joint probability mass function (PMF) estimation is a fundamental machine learning problem. The number of free parameters scales exponentially with respect to the number of random variables. Hence, most work on nonparametric PMF estimation is based on some structural assumptions such as clique factorization adopted by probabilistic graphical models, imposition of low rank on the joint probability…
▽ More
Joint probability mass function (PMF) estimation is a fundamental machine learning problem. The number of free parameters scales exponentially with respect to the number of random variables. Hence, most work on nonparametric PMF estimation is based on some structural assumptions such as clique factorization adopted by probabilistic graphical models, imposition of low rank on the joint probability tensor and reconstruction from 3-way or 2-way marginals, etc. In the present work, we link random projections of data to the problem of PMF estimation using ideas from tomography. We integrate this idea with the idea of low-rank tensor decomposition to show that we can estimate the joint density from just one-way marginals in a transformed space. We provide a novel algorithm for recovering factors of the tensor from one-way marginals, test it across a variety of synthetic and real-world datasets, and also perform MAP inference on the estimated model for classification.
△ Less
Submitted 24 March, 2021; v1 submitted 22 March, 2021;
originally announced March 2021.
-
SPOT: A framework for selection of prototypes using optimal transport
Authors:
Karthik S. Gurumoorthy,
Pratik Jawanpuria,
Bamdev Mishra
Abstract:
In this work, we develop an optimal transport (OT) based framework to select informative prototypical examples that best represent a given target dataset. Summarizing a given target dataset via representative examples is an important problem in several machine learning applications where human understanding of the learning models and underlying data distribution is essential for decision making. W…
▽ More
In this work, we develop an optimal transport (OT) based framework to select informative prototypical examples that best represent a given target dataset. Summarizing a given target dataset via representative examples is an important problem in several machine learning applications where human understanding of the learning models and underlying data distribution is essential for decision making. We model the prototype selection problem as learning a sparse (empirical) probability distribution having the minimum OT distance from the target distribution. The learned probability measure supported on the chosen prototypes directly corresponds to their importance in representing the target data. We show that our objective function enjoys a key property of submodularity and propose an efficient greedy method that is both computationally fast and possess deterministic approximation guarantees. Empirical results on several real world benchmarks illustrate the efficacy of our approach.
△ Less
Submitted 5 April, 2021; v1 submitted 18 March, 2021;
originally announced March 2021.
-
Think out of the package: Recommending package types for e-commerce shipments
Authors:
Karthik S. Gurumoorthy,
Subhajit Sanyal,
Vineet Chaoji
Abstract:
Multiple product attributes like dimensions, weight, fragility, liquid content etc. determine the package type used by e-commerce companies to ship products. Sub-optimal package types lead to damaged shipments, incurring huge damage related costs and adversely impacting the company's reputation for safe delivery. Items can be shipped in more protective packages to reduce damage costs, however this…
▽ More
Multiple product attributes like dimensions, weight, fragility, liquid content etc. determine the package type used by e-commerce companies to ship products. Sub-optimal package types lead to damaged shipments, incurring huge damage related costs and adversely impacting the company's reputation for safe delivery. Items can be shipped in more protective packages to reduce damage costs, however this increases the shipment costs due to expensive packaging and higher transportation costs. In this work, we propose a multi-stage approach that trades-off between shipment and damage costs for each product, and accurately assigns the optimal package type using a scalable, computationally efficient linear time algorithm. A simple binary search algorithm is presented to find the hyper-parameter that balances between the shipment and damage costs. Our approach when applied to choosing package type for Amazon shipments, leads to significant cost savings of tens of millions of dollars in emerging marketplaces, by decreasing both the overall shipment cost and the number of in-transit damages. Our algorithm is live and deployed in the production system where, package types for more than 130,000 products have been modified based on the model's recommendation, realizing a reduction in damage rate of 24%.
△ Less
Submitted 5 June, 2020;
originally announced June 2020.
-
Streaming Methods for Restricted Strongly Convex Functions with Applications to Prototype Selection
Authors:
Karthik S. Gurumoorthy,
Amit Dhurandhar
Abstract:
In this paper, we show that if the optimization function is restricted-strongly-convex (RSC) and restricted-smooth (RSM) -- a rich subclass of weakly submodular functions -- then a streaming algorithm with constant factor approximation guarantee is possible. More generally, our results are applicable to any monotone weakly submodular function with submodularity ratio bounded from above. This (posi…
▽ More
In this paper, we show that if the optimization function is restricted-strongly-convex (RSC) and restricted-smooth (RSM) -- a rich subclass of weakly submodular functions -- then a streaming algorithm with constant factor approximation guarantee is possible. More generally, our results are applicable to any monotone weakly submodular function with submodularity ratio bounded from above. This (positive) result which provides a sufficient condition for having a constant factor streaming guarantee for weakly submodular functions may be of special interest given the recent negative result (Elenberg et al., 2017) for the general class of weakly submodular functions. We apply our streaming algorithms for creating compact synopsis of large complex datasets, by selecting $m$ representative elements, by optimizing a suitable RSC and RSM objective function. Above results hold even with additional constraints such as learning non-negative weights, for interpretability, for each selected element indicative of its importance. We empirically evaluate our algorithms on two real datasets: MNIST- a handwritten digits dataset and Letters- a UCI dataset containing the alphabet written in different fonts and styles. We observe that our algorithms are orders of magnitude faster than the state-of-the-art streaming algorithm for weakly submodular functions and with our main algorithm still providing equally good solutions in practice.
△ Less
Submitted 21 July, 2018;
originally announced July 2018.
-
Signal Recovery in Perturbed Fourier Compressed Sensing
Authors:
Eeshan Malhotra,
Himanshu Pandotra,
Ajit Rajwade,
Karthik S. Gurumoorthy
Abstract:
In many applications in compressed sensing, the measurement matrix is a Fourier matrix, i.e., it measures the Fourier transform of the underlying signal at some specified `base' frequencies $\{u_i\}_{i=1}^M$, where $M$ is the number of measurements. However due to system calibration errors, the system may measure the Fourier transform at frequencies $\{u_i + δ_i\}_{i=1}^M$ that are different from…
▽ More
In many applications in compressed sensing, the measurement matrix is a Fourier matrix, i.e., it measures the Fourier transform of the underlying signal at some specified `base' frequencies $\{u_i\}_{i=1}^M$, where $M$ is the number of measurements. However due to system calibration errors, the system may measure the Fourier transform at frequencies $\{u_i + δ_i\}_{i=1}^M$ that are different from the base frequencies and where $\{δ_i\}_{i=1}^M$ are unknown. Ignoring perturbations of this nature can lead to major errors in signal recovery. In this paper, we present a simple but effective alternating minimization algorithm to recover the perturbations in the frequencies \emph{in situ} with the signal, which we assume is sparse or compressible in some known basis. In many cases, the perturbations $\{δ_i\}_{i=1}^M$ can be expressed in terms of a small number of unique parameters $P \ll M$. We demonstrate that in such cases, the method leads to excellent quality results that are several times better than baseline algorithms (which are based on existing off-grid methods in the recent literature on direction of arrival (DOA) estimation, modified to suit the computational problem in this paper). Our results are also robust to noise in the measurement values. We also provide theoretical results for (1) the convergence of our algorithm, and (2) the uniqueness of its solution under some restrictions.
△ Less
Submitted 8 February, 2018; v1 submitted 4 August, 2017;
originally announced August 2017.
-
Efficient Data Representation by Selecting Prototypes with Importance Weights
Authors:
Karthik S. Gurumoorthy,
Amit Dhurandhar,
Guillermo Cecchi,
Charu Aggarwal
Abstract:
Prototypical examples that best summarizes and compactly represents an underlying complex data distribution communicate meaningful insights to humans in domains where simple explanations are hard to extract. In this paper we present algorithms with strong theoretical guarantees to mine these data sets and select prototypes a.k.a. representatives that optimally describes them. Our work notably gene…
▽ More
Prototypical examples that best summarizes and compactly represents an underlying complex data distribution communicate meaningful insights to humans in domains where simple explanations are hard to extract. In this paper we present algorithms with strong theoretical guarantees to mine these data sets and select prototypes a.k.a. representatives that optimally describes them. Our work notably generalizes the recent work by Kim et al. (2016) where in addition to selecting prototypes, we also associate non-negative weights which are indicative of their importance. This extension provides a single coherent framework under which both prototypes and criticisms (i.e. outliers) can be found. Furthermore, our framework works for any symmetric positive definite kernel thus addressing one of the key open questions laid out in Kim et al. (2016). By establishing that our objective function enjoys a key property of that of weak submodularity, we present a fast ProtoDash algorithm and also derive approximation guarantees for the same. We demonstrate the efficacy of our method on diverse domains such as retail, digit recognition (MNIST) and on publicly available 40 health questionnaires obtained from the Center for Disease Control (CDC) website maintained by the US Dept. of Health. We validate the results quantitatively as well as qualitatively based on expert feedback and recently published scientific studies on public health, thus showcasing the power of our technique in providing actionability (for retail), utility (for MNIST) and insight (on CDC datasets) which arguably are the hallmarks of an effective data mining method.
△ Less
Submitted 12 August, 2019; v1 submitted 5 July, 2017;
originally announced July 2017.
-
Reconstruction Error Bounds for Compressed Sensing under Poisson or Poisson-Gaussian Noise Using Variance Stabilization Transforms
Authors:
Deepak Garg,
Pakshal Bohra,
Karthik S. Gurumoorthy,
Ajit Rajwade
Abstract:
Most existing bounds for signal reconstruction from compressive measurements make the assumption of additive signal-independent noise. However in many compressive imaging systems, the noise statistics are more accurately represented by Poisson or Poisson-Gaussian noise models. In this paper, we derive upper bounds for signal reconstruction error from compressive measurements which are corrupted by…
▽ More
Most existing bounds for signal reconstruction from compressive measurements make the assumption of additive signal-independent noise. However in many compressive imaging systems, the noise statistics are more accurately represented by Poisson or Poisson-Gaussian noise models. In this paper, we derive upper bounds for signal reconstruction error from compressive measurements which are corrupted by Poisson or Poisson-Gaussian noise. The features of our bounds are as follows: (1) The bounds are derived for a probabilistically motivated, computationally tractable convex estimator with principled parameter selection. The estimator penalizes signal sparsity subject to a constraint that imposes an upper bound on a term based on variance stabilization transforms to approximate the Poisson or Poisson-Gaussian negative log-likelihoods. (2) They are applicable to signals that are sparse as well as compressible in any orthonormal basis, and are derived for compressive systems obeying realistic constraints such as non-negativity and flux-preservation. We present extensive numerical results for signal reconstruction under varying number of measurements and varying signal intensity levels.
△ Less
Submitted 12 February, 2018; v1 submitted 3 July, 2017;
originally announced July 2017.
-
Degenerate Kalman filter error covariances and their convergence onto the unstable subspace
Authors:
Marc Bocquet,
Karthik S. Gurumoorthy,
Amit Apte,
Alberto Carrassi,
Colin Grudzien,
Christopher K. R. T. Jones
Abstract:
The characteristics of the model dynamics are critical in the performance of (ensemble) Kalman filters. In particular, as emphasized in the seminal work of Anna Trevisan and co-authors, the error covariance matrix is asymptotically supported by the unstable-neutral subspace only, i.e., it is spanned by the backward Lyapunov vectors with non-negative exponents. This behavior is at the core of algor…
▽ More
The characteristics of the model dynamics are critical in the performance of (ensemble) Kalman filters. In particular, as emphasized in the seminal work of Anna Trevisan and co-authors, the error covariance matrix is asymptotically supported by the unstable-neutral subspace only, i.e., it is spanned by the backward Lyapunov vectors with non-negative exponents. This behavior is at the core of algorithms known as Assimilation in the Unstable Subspace, although a formal proof was still missing.
This paper provides the analytical proof of the convergence of the Kalman filter covariance matrix onto the unstable-neutral subspace when the dynamics and the observation operator are linear and when the dynamical model is error-free, for any, possibly rank-deficient, initial error covariance matrix. The rate of convergence is provided as well. The derivation is based on an expression that explicitly relates the error covariances at an arbitrary time to the initial ones. It is also shown that if the unstable and neutral directions of the model are sufficiently observed and if the column space of the initial covariance matrix has a non-zero projection onto all of the forward Lyapunov vectors associated with the unstable and neutral directions of the dynamics, the covariance matrix of the Kalman filter collapses onto an asymptotic sequence which is independent of the initial covariances. Numerical results are also shown to illustrate and support the theoretical findings.
△ Less
Submitted 31 March, 2017; v1 submitted 9 April, 2016;
originally announced April 2016.
-
Sensitivity Analysis for additive STDP rule
Authors:
Subhajit Sengupta,
Karthik S. Gurumoorthy,
Arunava Banerjee
Abstract:
Spike Timing Dependent Plasticity (STDP) is a Hebbian like synaptic learning rule. The basis of STDP has strong experimental evidences and it depends on precise input and output spike timings. In this paper we show that under biologically plausible spiking regime, slight variability in the spike timing leads to drastically different evolution of synaptic weights when its dynamics are governed by t…
▽ More
Spike Timing Dependent Plasticity (STDP) is a Hebbian like synaptic learning rule. The basis of STDP has strong experimental evidences and it depends on precise input and output spike timings. In this paper we show that under biologically plausible spiking regime, slight variability in the spike timing leads to drastically different evolution of synaptic weights when its dynamics are governed by the additive STDP rule.
△ Less
Submitted 13 April, 2015; v1 submitted 28 February, 2015;
originally announced March 2015.
-
Rank deficiency of Kalman error covariance matrices in linear time-varying system with deterministic evolution
Authors:
Karthik S. Gurumoorthy,
Colin Grudzien,
Amit Apte,
Alberto Carrassi,
Christopher K. R. T. Jones
Abstract:
We prove that for linear, discrete, time-varying, deterministic system (perfect model) with noisy outputs, the Riccati transformation in the Kalman filter asymptotically bounds the rank of the forecast and the analysis error covariance matrices to be less than or equal to the number of non-negative Lyapunov exponents of the system. Further, the support of these error covariance matrices is shown t…
▽ More
We prove that for linear, discrete, time-varying, deterministic system (perfect model) with noisy outputs, the Riccati transformation in the Kalman filter asymptotically bounds the rank of the forecast and the analysis error covariance matrices to be less than or equal to the number of non-negative Lyapunov exponents of the system. Further, the support of these error covariance matrices is shown to be confined to the space spanned by the unstable-neutral backward Lyapunov vectors, providing the theoretical justification for the methodology of the algorithms that perform assimilation only in the unstable-neutral subspace. The equivalent property of the autonomous system is investigated as a special case.
△ Less
Submitted 4 October, 2016; v1 submitted 17 March, 2015;
originally announced March 2015.
-
Error bounds for gradient density estimation computed from a finite sample set using the method of stationary phase
Authors:
Karthik S. Gurumoorthy,
Anand Rangarajan
Abstract:
For a twice continuously differentiable function $S$, we define the density function of its gradient (derivative in one dimension) $s = S^{\prime}$ as a random variable transformation of a uniformly distributed random variable using $s$ as the transformation function. Given $N$ values of $S$ sampled at equally spaced locations, we demonstrate using the method of stationary phase that the approxima…
▽ More
For a twice continuously differentiable function $S$, we define the density function of its gradient (derivative in one dimension) $s = S^{\prime}$ as a random variable transformation of a uniformly distributed random variable using $s$ as the transformation function. Given $N$ values of $S$ sampled at equally spaced locations, we demonstrate using the method of stationary phase that the approximation error between the integral of the scaled, discrete power spectrum of the wave function $φ^{D}_τ=\frac{1}{\sqrt{L}}\exp\left(\frac{iS}τ\right)$ and the integral of the true density function of $s$ over an arbitrarily small interval is bounded above by $O(1/N)$ as $N \rightarrow \infty$ ($τ\rightarrow 0$). In addition to its easy implementation and fast computability in $O(N \log N)$ that only requires computing the discrete Fourier transform, our framework for obtaining the derivative density does not involve any parameter selection like the number of histogram bins, width of the histogram bins, width of the kernel parameter, number of mixture components etc. as required by other widely applied methods like histograms and Parzen windows.
△ Less
Submitted 29 December, 2016; v1 submitted 3 April, 2014;
originally announced April 2014.
-
A fast eikonal equation solver using the Schrodinger wave equation
Authors:
Karthik S. Gurumoorthy,
Adrian M. Peter,
Birmingham Hang Guan,
Anand Rangarajan
Abstract:
We use a Schrödinger wave equation formalism to solve the eikonal equation. In our framework, a solution to the eikonal equation is obtained in the limit as Planck's constant $\hbar$ (treated as a free parameter) tends to zero of the solution to the corresponding linear Schrödinger equation. The Schrödinger equation corresponding to the eikonal turns out to be a \emph{generalized, screened Poisson…
▽ More
We use a Schrödinger wave equation formalism to solve the eikonal equation. In our framework, a solution to the eikonal equation is obtained in the limit as Planck's constant $\hbar$ (treated as a free parameter) tends to zero of the solution to the corresponding linear Schrödinger equation. The Schrödinger equation corresponding to the eikonal turns out to be a \emph{generalized, screened Poisson equation}. Despite being linear, it does not have a closed-form solution for arbitrary forcing functions. We present two different techniques to solve the screened Poisson equation. In the first approach we use a standard perturbation analysis approach to derive a new algorithm which is guaranteed to converge provided the forcing function is bounded and positive. The perturbation technique requires a sequence of discrete convolutions which can be performed in $O(N\log N)$ using the Fast Fourier Transform (FFT) where $N$ is the number of grid points. In the second method we discretize the linear Laplacian operator by the finite difference method leading to a sparse linear system of equations which can be solved using the plethora of sparse solvers. The eikonal solution is recovered from the exponent of the resultant scalar field. Our approach eliminates the need to explicitly construct viscosity solutions as customary with direct solutions to the eikonal. Since the linear equation is computed for a small but non-zero $\hbar$, the obtained solution is an approximation. Though our solution framework is applicable to the general class of eikonal problems, we detail specifics for the popular vision applications of shape-from-shading, vessel segmentation, and path planning.
△ Less
Submitted 8 February, 2015; v1 submitted 8 March, 2014;
originally announced March 2014.
-
On the dynamic compressibility of sets
Authors:
Karthik S. Gurumoorthy
Abstract:
We define a new notion of compressibility of a set of numbers through the dynamics of a polynomial function. We provide approaches to solve the problem by reducing it to the multi-criteria traveling salesman problem through a series of transformations. We then establish computational complexity results by giving some NP-completeness proofs. We also discuss about a notion of $ε$ K-compressibility o…
▽ More
We define a new notion of compressibility of a set of numbers through the dynamics of a polynomial function. We provide approaches to solve the problem by reducing it to the multi-criteria traveling salesman problem through a series of transformations. We then establish computational complexity results by giving some NP-completeness proofs. We also discuss about a notion of $ε$ K-compressibility of a set, with regard to lossy compression and deduce the necessary condition for the given set to be $ε$ K-compressible. Finally, we conclude by providing a list of open problems solutions to which could extend the applicability the our technique.
△ Less
Submitted 3 February, 2013;
originally announced February 2013.
-
Gradient density estimation in arbitrary finite dimensions using the method of stationary phase
Authors:
Karthik S. Gurumoorthy,
Anand Rangarajan,
John Corring
Abstract:
We prove that the density function of the gradient of a sufficiently smooth function $S : Ω\subset \mathbb{R}^d \rightarrow \mathbb{R}$, obtained via a random variable transformation of a uniformly distributed random variable, is increasingly closely approximated by the normalized power spectrum of $φ=\exp\left(\frac{iS}τ\right)$ as the free parameter $τ\rightarrow 0$. The result is shown using th…
▽ More
We prove that the density function of the gradient of a sufficiently smooth function $S : Ω\subset \mathbb{R}^d \rightarrow \mathbb{R}$, obtained via a random variable transformation of a uniformly distributed random variable, is increasingly closely approximated by the normalized power spectrum of $φ=\exp\left(\frac{iS}τ\right)$ as the free parameter $τ\rightarrow 0$. The result is shown using the stationary phase approximation and standard integration techniques and requires proper ordering of limits. We highlight a relationship with the well-known characteristic function approach to density estimation, and detail why our result is distinct from this approach.
△ Less
Submitted 4 September, 2014; v1 submitted 13 November, 2012;
originally announced November 2012.
-
A new variational principle for the Euclidean distance function: Linear approach to the non-linear eikonal problem
Authors:
Karthik S. Gurumoorthy,
Anand Rangarajan
Abstract:
We present a fast convolution-based technique for computing an approximate, signed Euclidean distance function $S$ on a set of 2D and 3D grid locations. Instead of solving the non-linear, static Hamilton-Jacobi equation ($\|\nabla S\|=1$), our solution stems from first solving for a scalar field $φ$ in a linear differential equation and then deriving the solution for $S$ by taking the negative log…
▽ More
We present a fast convolution-based technique for computing an approximate, signed Euclidean distance function $S$ on a set of 2D and 3D grid locations. Instead of solving the non-linear, static Hamilton-Jacobi equation ($\|\nabla S\|=1$), our solution stems from first solving for a scalar field $φ$ in a linear differential equation and then deriving the solution for $S$ by taking the negative logarithm. In other words, when $S$ and $φ$ are related by $φ= \exp \left(-\frac{S}τ \right)$ and $φ$ satisfies a specific linear differential equation corresponding to the extremum of a variational problem, we obtain the approximate Euclidean distance function $S = -τ\log(φ)$ which converges to the true solution in the limit as $τ\rightarrow 0$. This is in sharp contrast to techniques like the fast marching and fast sweeping methods which directly solve the Hamilton-Jacobi equation by the Godunov upwind discretization scheme. Our linear formulation results in a closed-form solution to the approximate Euclidean distance function expressible as a discrete convolution, and hence efficiently computable using the fast Fourier transform (FFT). Our solution also circumvents the need for spatial discretization of the derivative operator. As $τ\rightarrow0$ we show the convergence of our results to the true solution and also bound the error for a given value of $τ$. The differentiability of our solution allows us to compute---using a set of convolutions---the first and second derivatives of the approximate distance function. In order to determine the sign of the distance function (defined to be positive inside a closed region and negative outside), we compute the winding number in 2D and the topological degree in 3D, whose computations can also be performed via fast convolutions. We demonstrate the efficacy of our method through a set of experimental results.
△ Less
Submitted 8 February, 2015; v1 submitted 13 December, 2011;
originally announced December 2011.
-
An application of the stationary phase method for estimating probability densities of function derivatives
Authors:
Karthik S. Gurumoorthy,
Anand Rangarajan,
Arunava Banerjee
Abstract:
We prove a novel result wherein the density function of the gradients---corresponding to density function of the derivatives in one dimension---of a thrice differentiable function S (obtained via a random variable transformation of a uniformly distributed random variable) defined on a closed, bounded interval Ω\subset R is accurately approximated by the normalized power spectrum of φ=exp(iS/τ) as…
▽ More
We prove a novel result wherein the density function of the gradients---corresponding to density function of the derivatives in one dimension---of a thrice differentiable function S (obtained via a random variable transformation of a uniformly distributed random variable) defined on a closed, bounded interval Ω\subset R is accurately approximated by the normalized power spectrum of φ=exp(iS/τ) as the free parameter τ-->0. The result is shown using the well known stationary phase approximation and standard integration techniques and requires proper ordering of limits. Experimental results provide anecdotal visual evidence corroborating the result.
△ Less
Submitted 3 February, 2013; v1 submitted 8 August, 2011;
originally announced August 2011.
-
Distance Transform Gradient Density Estimation using the Stationary Phase Approximation
Authors:
Karthik S. Gurumoorthy,
Anand Rangarajan
Abstract:
The complex wave representation (CWR) converts unsigned 2D distance transforms into their corresponding wave functions. Here, the distance transform S(X) appears as the phase of the wave function φ(X)---specifically, φ(X)=exp(iS(X)/τwhere τis a free parameter. In this work, we prove a novel result using the higher-order stationary phase approximation: we show convergence of the normalized power sp…
▽ More
The complex wave representation (CWR) converts unsigned 2D distance transforms into their corresponding wave functions. Here, the distance transform S(X) appears as the phase of the wave function φ(X)---specifically, φ(X)=exp(iS(X)/τwhere τis a free parameter. In this work, we prove a novel result using the higher-order stationary phase approximation: we show convergence of the normalized power spectrum (squared magnitude of the Fourier transform) of the wave function to the density function of the distance transform gradients as the free parameter τ-->0. In colloquial terms, spatial frequencies are gradient histogram bins. Since the distance transform gradients have only orientation information (as their magnitudes are identically equal to one almost everywhere), as τ-->0, the 2D Fourier transform values mainly lie on the unit circle in the spatial frequency domain. The proof of the result involves standard integration techniques and requires proper ordering of limits. Our mathematical relation indicates that the CWR of distance transforms is an intriguing, new representation.
△ Less
Submitted 5 February, 2013; v1 submitted 18 April, 2011;
originally announced April 2011.
