-
KAN-Mixers: a new deep learning architecture for image classification
Authors:
Jorge Luiz dos Santos Canuto,
Linnyer Beatrys Ruiz Aylon,
Rodrigo Clemente Thom de Souza
Abstract:
Due to their effective performance, Convolutional Neural Network (CNN) and Vision Transformer (ViT) architectures have become the standard for solving computer vision tasks. Such architectures require large data sets and rely on convolution and self-attention operations. In 2021, MLP-Mixer emerged, an architecture that relies only on Multilayer Perceptron (MLP) and achieves extremely competitive r…
▽ More
Due to their effective performance, Convolutional Neural Network (CNN) and Vision Transformer (ViT) architectures have become the standard for solving computer vision tasks. Such architectures require large data sets and rely on convolution and self-attention operations. In 2021, MLP-Mixer emerged, an architecture that relies only on Multilayer Perceptron (MLP) and achieves extremely competitive results when compared to CNNs and ViTs. Despite its good performance in computer vision tasks, the MLP-Mixer architecture may not be suitable for refined feature extraction in images. Recently, the Kolmogorov-Arnold Network (KAN) was proposed as a promising alternative to MLP models. KANs promise to improve accuracy and interpretability when compared to MLPs. Therefore, the present work aims to design a new mixer-based architecture, called KAN-Mixers, using KANs as main layers and evaluate its performance, in terms of several performance metrics, in the image classification task. As main results obtained, the KAN-Mixers model was superior to the MLP, MLP-Mixer and KAN models in the Fashion-MNIST and CIFAR-10 datasets, with 0.9030 and 0.6980 of average accuracy, respectively.
△ Less
Submitted 11 March, 2025;
originally announced March 2025.
-
Bias-corrected estimator for intrinsic dimension and differential entropy--a visual multiscale approach
Authors:
Jugurta Montalvão,
Jânio Canuto,
Luiz Miranda
Abstract:
Intrinsic dimension and differential entropy estimators are studied in this paper, including their systematic bias. A pragmatic approach for joint estimation and bias correction of these two fundamental measures is proposed. Shared steps on both estimators are highlighted, along with their useful consequences to data analysis. It is shown that both estimators can be complementary parts of a single…
▽ More
Intrinsic dimension and differential entropy estimators are studied in this paper, including their systematic bias. A pragmatic approach for joint estimation and bias correction of these two fundamental measures is proposed. Shared steps on both estimators are highlighted, along with their useful consequences to data analysis. It is shown that both estimators can be complementary parts of a single approach, and that the simultaneous estimation of differential entropy and intrinsic dimension give meaning to each other, where estimates at different observation scales convey different perspectives of underlying manifolds. Experiments with synthetic and real datasets are presented to illustrate how to extract meaning from visual inspections, and how to compensate for biases.
△ Less
Submitted 29 April, 2020;
originally announced April 2020.
-
Offline signature authenticity verification through unambiguously connected skeleton segments
Authors:
Jugurta Montalvão,
Luiz Miranda,
Jânio Canuto
Abstract:
A method for offline signature verification is presented in this paper. It is based on the segmentation of the signature skeleton (through standard image skeletonization) into unambiguous sequences of points, or unambiguously connected skeleton segments corresponding to vectorial representations of signature portions. These segments are assumed to be the fundamental carriers of useful information…
▽ More
A method for offline signature verification is presented in this paper. It is based on the segmentation of the signature skeleton (through standard image skeletonization) into unambiguous sequences of points, or unambiguously connected skeleton segments corresponding to vectorial representations of signature portions. These segments are assumed to be the fundamental carriers of useful information for authenticity verification, and are compactly encoded as sets of 9 scalars (4 sampled coordinates and 1 length measure). Thus signature authenticity is inferred through Euclidean distance based comparisons between pairs of such compact representations. The average performance of this method is evaluated through experiments with offline versions of signatures from the MCYT-100 database. For comparison purposes, three other approaches are applied to the same set of signatures, namely: (1) a straightforward approach based on Dynamic Time Warping distances between segments, (2) a published method by [shanker2007], also based on DTW, and (3) the average human performance under equivalent experimental protocol. Results suggest that if human performance is taken as a goal for automatic verification, then we should discard signature shape details to approach this goal. Moreover, our best result -- close to human performance -- was obtained by the simplest strategy, where equal weights were given to segment shape and length.
△ Less
Submitted 8 November, 2017;
originally announced November 2017.
