-
No-Trick (Treat) Kernel Adaptive Filtering using Deterministic Features
Authors:
Kan Li,
Jose C. Principe
Abstract:
Kernel methods form a powerful, versatile, and theoretically-grounded unifying framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the kernel trick to perform pairwise evaluations of a kernel function, which leads to scalability issues for large datasets due to its linear and superlinear growth with respect to the training data. A popula…
▽ More
Kernel methods form a powerful, versatile, and theoretically-grounded unifying framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the kernel trick to perform pairwise evaluations of a kernel function, which leads to scalability issues for large datasets due to its linear and superlinear growth with respect to the training data. A popular approach to tackle this problem, known as random Fourier features (RFFs), samples from a distribution to obtain the data-independent basis of a higher finite-dimensional feature space, where its dot product approximates the kernel function. Recently, deterministic, rather than random construction has been shown to outperform RFFs, by approximating the kernel in the frequency domain using Gaussian quadrature. In this paper, we view the dot product of these explicit mappings not as an approximation, but as an equivalent positive-definite kernel that induces a new finite-dimensional reproducing kernel Hilbert space (RKHS). This opens the door to no-trick (NT) online kernel adaptive filtering (KAF) that is scalable and robust. Random features are prone to large variances in performance, especially for smaller dimensions. Here, we focus on deterministic feature-map construction based on polynomial-exact solutions and show their superiority over random constructions. Without loss of generality, we apply this approach to classical adaptive filtering algorithms and validate the methodology to show that deterministic features are faster to generate and outperform state-of-the-art kernel methods based on random Fourier features.
△ Less
Submitted 10 December, 2019;
originally announced December 2019.
-
An Extended Result on the Optimal Estimation under Minimum Error Entropy Criterion
Authors:
Badong Chen,
Guangmin Wang,
Nanning Zheng,
Jose C. Principe
Abstract:
The minimum error entropy (MEE) criterion has been successfully used in fields such as parameter estimation, system identification and the supervised machine learning. There is in general no explicit expression for the optimal MEE estimate unless some constraints on the conditional distribution are imposed. A recent paper has proved that if the conditional density is conditionally symmetric and un…
▽ More
The minimum error entropy (MEE) criterion has been successfully used in fields such as parameter estimation, system identification and the supervised machine learning. There is in general no explicit expression for the optimal MEE estimate unless some constraints on the conditional distribution are imposed. A recent paper has proved that if the conditional density is conditionally symmetric and unimodal (CSUM), then the optimal MEE estimate (with Shannon entropy) equals the conditional median. In this study, we extend this result to the generalized MEE estimation where the optimality criterion is the Renyi entropy or equivalently, the α-order information potential (IP).
△ Less
Submitted 24 January, 2014;
originally announced January 2014.
-
Approximate reconstruction of bandlimited functions for the integrate and fire sampler
Authors:
Hans G. Feichtinger,
José C. Príncipe,
José Luis Romero,
Alexander Singh Alvarado,
Gino Angelo Velasco
Abstract:
In this paper we study the reconstruction of a bandlimited signal from samples generated by the integrate and fire model. This sampler allows us to trade complexity in the reconstruction algorithms for simple hardware implementations, and is specially convenient in situations where the sampling device is limited in terms of power, area and bandwidth.
Although perfect reconstruction for this sa…
▽ More
In this paper we study the reconstruction of a bandlimited signal from samples generated by the integrate and fire model. This sampler allows us to trade complexity in the reconstruction algorithms for simple hardware implementations, and is specially convenient in situations where the sampling device is limited in terms of power, area and bandwidth.
Although perfect reconstruction for this sampler is impossible, we give a general approximate reconstruction procedure and bound the corresponding error. We also show the performance of the proposed algorithm through numerical simulations.
△ Less
Submitted 3 December, 2009;
originally announced December 2009.
