-
Lightweight Attribute Localizing Models for Pedestrian Attribute Recognition
Authors:
Ashish Jha,
Dimitrii Ermilov,
Konstantin Sobolev,
Anh Huy Phan,
Salman Ahmadi-Asl,
Naveed Ahmed,
Imran Junejo,
Zaher AL Aghbari,
Thar Baker,
Ahmed Mohamed Khedr,
Andrzej Cichocki
Abstract:
Pedestrian Attribute Recognition (PAR) deals with the problem of identifying features in a pedestrian image. It has found interesting applications in person retrieval, suspect re-identification and soft biometrics. In the past few years, several Deep Neural Networks (DNNs) have been designed to solve the task; however, the developed DNNs predominantly suffer from over-parameterization and high com…
▽ More
Pedestrian Attribute Recognition (PAR) deals with the problem of identifying features in a pedestrian image. It has found interesting applications in person retrieval, suspect re-identification and soft biometrics. In the past few years, several Deep Neural Networks (DNNs) have been designed to solve the task; however, the developed DNNs predominantly suffer from over-parameterization and high computational complexity. These problems hinder them from being exploited in resource-constrained embedded devices with limited memory and computational capacity. By reducing a network's layers using effective compression techniques, such as tensor decomposition, neural network compression is an effective method to tackle these problems. We propose novel Lightweight Attribute Localizing Models (LWALM) for Pedestrian Attribute Recognition (PAR). LWALM is a compressed neural network obtained after effective layer-wise compression of the Attribute Localization Model (ALM) using the Canonical Polyadic Decomposition with Error Preserving Correction (CPD-EPC) algorithm.
△ Less
Submitted 16 June, 2023;
originally announced June 2023.
-
How to Train Unstable Looped Tensor Network
Authors:
Anh-Huy Phan,
Konstantin Sobolev,
Dmitry Ermilov,
Igor Vorona,
Nikolay Kozyrskiy,
Petr Tichavsky,
Andrzej Cichocki
Abstract:
A rising problem in the compression of Deep Neural Networks is how to reduce the number of parameters in convolutional kernels and the complexity of these layers by low-rank tensor approximation. Canonical polyadic tensor decomposition (CPD) and Tucker tensor decomposition (TKD) are two solutions to this problem and provide promising results. However, CPD often fails due to degeneracy, making the…
▽ More
A rising problem in the compression of Deep Neural Networks is how to reduce the number of parameters in convolutional kernels and the complexity of these layers by low-rank tensor approximation. Canonical polyadic tensor decomposition (CPD) and Tucker tensor decomposition (TKD) are two solutions to this problem and provide promising results. However, CPD often fails due to degeneracy, making the networks unstable and hard to fine-tune. TKD does not provide much compression if the core tensor is big. This motivates using a hybrid model of CPD and TKD, a decomposition with multiple Tucker models with small core tensor, known as block term decomposition (BTD). This paper proposes a more compact model that further compresses the BTD by enforcing core tensors in BTD identical. We establish a link between the BTD with shared parameters and a looped chain tensor network (TC). Unfortunately, such strongly constrained tensor networks (with loop) encounter severe numerical instability, as proved by y (Landsberg, 2012) and (Handschuh, 2015a). We study perturbation of chain tensor networks, provide interpretation of instability in TC, demonstrate the problem. We propose novel methods to gain the stability of the decomposition results, keep the network robust and attain better approximation. Experimental results will confirm the superiority of the proposed methods in compression of well-known CNNs, and TC decomposition under challenging scenarios
△ Less
Submitted 4 March, 2022;
originally announced March 2022.
-
Interpolation error of misspecified Gaussian process regression
Authors:
A. Zaytsev,
E. Romanenkova,
D. Ermilov
Abstract:
An interpolation error is an integral of the squared error of a regression model over a domain of interest. We consider the interpolation error for the case of misspecified Gaussian process regression: used covariance function differs from the true one. We derive the interpolation error for an infinite grid design of experiments. In particular, we show that for Matern 1/2 covariance function poor…
▽ More
An interpolation error is an integral of the squared error of a regression model over a domain of interest. We consider the interpolation error for the case of misspecified Gaussian process regression: used covariance function differs from the true one. We derive the interpolation error for an infinite grid design of experiments. In particular, we show that for Matern 1/2 covariance function poor estimation of parameters only slightly affects the quality of interpolation. Then we proceed to numerical experiments that consider the misspecification for the most common covariance functions including other Matern and squared exponential covariance functions. For them, the quality of estimates of parameters affects the interpolation error.
△ Less
Submitted 26 March, 2018;
originally announced March 2018.
