Sparse Linear Regression with Constraints: A Flexible Entropy-based Framework
Authors:
Amber Srivastava,
Alisina Bayati,
Srinivasa Salapaka
Abstract:
This work presents a new approach to solve the sparse linear regression problem, i.e., to determine a k-sparse vector w in R^d that minimizes the cost ||y - Aw||^2_2. In contrast to the existing methods, our proposed approach splits this k-sparse vector into two parts -- (a) a column stochastic binary matrix V, and (b) a vector x in R^k. Here, the binary matrix V encodes the location of the k non-…
▽ More
This work presents a new approach to solve the sparse linear regression problem, i.e., to determine a k-sparse vector w in R^d that minimizes the cost ||y - Aw||^2_2. In contrast to the existing methods, our proposed approach splits this k-sparse vector into two parts -- (a) a column stochastic binary matrix V, and (b) a vector x in R^k. Here, the binary matrix V encodes the location of the k non-zero entries in w. Equivalently, it encodes the subset of k columns in the matrix A that map w to y. We demonstrate that this enables modeling several non-trivial application-specific structural constraints on w as constraints on V. The vector x comprises of the actual non-zero values in w. We use Maximum Entropy Principle (MEP) to solve the resulting optimization problem. In particular, we ascribe a probability distribution to the set of all feasible binary matrices V, and iteratively determine this distribution and the vector x such that the associated Shannon entropy gets minimized, and the regression cost attains a pre-specified value. The resulting algorithm employs homotopy from the convex entropy function to the non-convex cost function to avoid poor local minimum. We demonstrate the efficacy and flexibility of our proposed approach in incorporating a variety of practical constraints, that are otherwise difficult to model using the existing benchmark methods.
△ Less
Submitted 17 November, 2023; v1 submitted 14 November, 2023;
originally announced November 2023.
Towards Efficient Modularity in Industrial Drying: A Combinatorial Optimization Viewpoint
Authors:
Alisina Bayati,
Amber Srivastava,
Amir Malvandi,
Hao Feng,
Srinivasa Salapaka
Abstract:
The industrial drying process consumes approximately 12% of the total energy used in manufacturing, with the potential for a 40% reduction in energy usage through improved process controls and the development of new drying technologies. To achieve cost-efficient and high-performing drying, multiple drying technologies can be combined in a modular fashion with optimal sequencing and control paramet…
▽ More
The industrial drying process consumes approximately 12% of the total energy used in manufacturing, with the potential for a 40% reduction in energy usage through improved process controls and the development of new drying technologies. To achieve cost-efficient and high-performing drying, multiple drying technologies can be combined in a modular fashion with optimal sequencing and control parameters for each. This paper presents a mathematical formulation of this optimization problem and proposes a framework based on the Maximum Entropy Principle (MEP) to simultaneously solve for both optimal values of control parameters and optimal sequence. The proposed algorithm addresses the combinatorial optimization problem with a non-convex cost function riddled with multiple poor local minima. Simulation results on drying distillers dried grain (DDG) products show up to 12% improvement in energy consumption compared to the most efficient single-stage drying process. The proposed algorithm converges to local minima and is designed heuristically to reach the global minimum.
△ Less
Submitted 5 April, 2023; v1 submitted 4 October, 2022;
originally announced October 2022.
