Showing 1–21 of 21 results for author: Andrecut, M

Residual Random Neural Networks

Authors: M. Andrecut

Abstract: The single-layer feedforward neural network with random weights is a recurring motif in the neural networks literature. The advantage of these networks is their simplified training, which reduces to solving a ridge-regression problem. A general assumption is that these networks require a large number of hidden neurons relative to the dimensionality of the data samples, in order to achieve good cla… ▽ More The single-layer feedforward neural network with random weights is a recurring motif in the neural networks literature. The advantage of these networks is their simplified training, which reduces to solving a ridge-regression problem. A general assumption is that these networks require a large number of hidden neurons relative to the dimensionality of the data samples, in order to achieve good classification accuracy. Contrary to this assumption, here we show that one can obtain good classification results even if the number of hidden neurons has the same order of magnitude as the dimensionality of the data samples, if this dimensionality is reasonably high. Inspired by this result, we also develop an efficient iterative residual training method for such random neural networks, and we extend the algorithm to the least-squares kernel version of the neural network model. Moreover, we also describe an encryption (obfuscation) method which can be used to protect both the data and the resulted network model. △ Less

Submitted 21 February, 2025; v1 submitted 25 October, 2024; originally announced October 2024.

Comments: revised version, 17 pages, 8 figures, added kernel method, added kernel type

Heuristic Optimal Transport in Branching Networks

Authors: M. Andrecut

Abstract: Optimal transport aims to learn a mapping of sources to targets by minimizing the cost, which is typically defined as a function of distance. The solution to this problem consists of straight line segments optimally connecting sources to targets, and it does not exhibit branching. These optimal solutions are in stark contrast with both natural, and man-made transportation networks, where branching… ▽ More Optimal transport aims to learn a mapping of sources to targets by minimizing the cost, which is typically defined as a function of distance. The solution to this problem consists of straight line segments optimally connecting sources to targets, and it does not exhibit branching. These optimal solutions are in stark contrast with both natural, and man-made transportation networks, where branching structures are prevalent. Here we discuss a fast heuristic branching method for optimal transport in networks. We also provide several numerical applications to synthetic examples, a simplified cardiovascular network, and the "Santa Claus" distribution network which includes 141,182 cities around the world, with known location and population. △ Less

Submitted 7 February, 2024; v1 submitted 11 November, 2023; originally announced November 2023.

Comments: Accepted in Int. J. Mod. Phys. C, 11 pages, 6 figures

ELM Ridge Regression Boosting

Authors: M. Andrecut

Abstract: We discuss a boosting approach for the Ridge Regression (RR) method, with applications to the Extreme Learning Machine (ELM), and we show that the proposed method significantly improves the classification performance and robustness of ELMs. We discuss a boosting approach for the Ridge Regression (RR) method, with applications to the Extreme Learning Machine (ELM), and we show that the proposed method significantly improves the classification performance and robustness of ELMs. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Comments: 6 pages, 2 figures

TensorFlow Chaotic Prediction and Blow Up

Authors: M. Andrecut

Abstract: Predicting the dynamics of chaotic systems is one of the most challenging tasks for neural networks, and machine learning in general. Here we aim to predict the spatiotemporal chaotic dynamics of a high-dimensional non-linear system. In our attempt we use the TensorFlow library, representing the state of the art for deep neural networks training and prediction. While our results are encouraging, a… ▽ More Predicting the dynamics of chaotic systems is one of the most challenging tasks for neural networks, and machine learning in general. Here we aim to predict the spatiotemporal chaotic dynamics of a high-dimensional non-linear system. In our attempt we use the TensorFlow library, representing the state of the art for deep neural networks training and prediction. While our results are encouraging, and show that the dynamics of the considered system can be predicted for short time, we also indirectly discovered an unexpected and undesirable behavior of the TensorFlow library. More specifically, the longer term prediction of the system's chaotic behavior quickly deteriorates and blows up due to the nondeterministic behavior of the TensorFlow library. Here we provide numerical evidence of the short time prediction ability, and of the longer term predictability blow up. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Comments: 10 pages, 3 figures

Autoencoders as Pattern Filters

Authors: M. Andrecut

Abstract: We discuss a simple approach to transform autoencoders into "pattern filters". Besides filtering, we show how this simple approach can be used also to build robust classifiers, by learning to filter only patterns of a given class. We discuss a simple approach to transform autoencoders into "pattern filters". Besides filtering, we show how this simple approach can be used also to build robust classifiers, by learning to filter only patterns of a given class. △ Less

Submitted 26 February, 2023; originally announced February 2023.

Comments: 10 pages, 5 figures

Secret sharing scheme based on hashing

Authors: M. Andrecut

Abstract: We propose an adaptive threshold multi secret sharing scheme based solely on cryptographically secure hash functions. We show that the proposed scheme is also: perfect, ideal, verifiable, and proactive. Moreover the proposed scheme has a low computational complexity comparing to the most common schemes operating over finite fields. We propose an adaptive threshold multi secret sharing scheme based solely on cryptographically secure hash functions. We show that the proposed scheme is also: perfect, ideal, verifiable, and proactive. Moreover the proposed scheme has a low computational complexity comparing to the most common schemes operating over finite fields. △ Less

Submitted 5 February, 2023; originally announced February 2023.

Comments: 7 pages

Growing Random Strings in CA

Authors: M. Andrecut

Abstract: We discuss a class of cellular automata (CA) able to produce long random strings, starting from short "seed" strings. The approach uses two principles borrowed from cryptography: diffusion and confusion. We show numerically that the strings are pseudo-random using three approaches based on: Fourier transform, entropy estimation, and compression. An application to cryptography is also included with… ▽ More We discuss a class of cellular automata (CA) able to produce long random strings, starting from short "seed" strings. The approach uses two principles borrowed from cryptography: diffusion and confusion. We show numerically that the strings are pseudo-random using three approaches based on: Fourier transform, entropy estimation, and compression. An application to cryptography is also included with the corresponding Python code. △ Less

Submitted 14 November, 2022; originally announced November 2022.

Comments: 9 pages, 4 figures

Attack vs Benign Network Intrusion Traffic Classification

Authors: M. Andrecut

Abstract: Intrusion detection systems (IDS) are used to monitor networks or systems for attack activity or policy violations. Such a system should be able to successfully identify anomalous deviations from normal traffic behavior. Here we discuss the machine learning approach to building an anomaly-based IDS using the CSE-CIC-IDS2018 dataset. Since the publication of this dataset a relatively large number o… ▽ More Intrusion detection systems (IDS) are used to monitor networks or systems for attack activity or policy violations. Such a system should be able to successfully identify anomalous deviations from normal traffic behavior. Here we discuss the machine learning approach to building an anomaly-based IDS using the CSE-CIC-IDS2018 dataset. Since the publication of this dataset a relatively large number of papers have been published, most of them presenting IDS architectures and results based on complex machine learning methods, like deep neural networks, gradient boosting classifiers, or hidden Markov models. Here we show that similar results can be obtained using a very simple nearest neighbor classification approach, avoiding the inherent complications of training such complex models. The advantages of the nearest neighbor algorithm are: (1) it is very simple to implement; (2) it is extremely robust; (3) it has no parameters, and therefore it cannot overfit the data. This result also shows that currently there is a trend of developing over-engineered solutions in the machine learning community. Such solutions are based on complex methods, like deep learning neural networks, without even considering baseline solutions corresponding to simple, but efficient methods. △ Less

Submitted 15 May, 2022; originally announced May 2022.

Comments: 8 pages

Sandbox Sample Classification Using Behavioral Indicators of Compromise

Authors: M. Andrecut

Abstract: Behavioral Indicators of Compromise are associated with various automated methods used to extract the sample behavior by observing the system function calls performed in a virtual execution environment. Thus, every sample is described by a set of BICs triggered by the sample behavior in the sandbox environment. Here we discuss a Machine Learning approach to the classification of the sandbox sample… ▽ More Behavioral Indicators of Compromise are associated with various automated methods used to extract the sample behavior by observing the system function calls performed in a virtual execution environment. Thus, every sample is described by a set of BICs triggered by the sample behavior in the sandbox environment. Here we discuss a Machine Learning approach to the classification of the sandbox samples as MALICIOUS or BENIGN, based on the list of triggered BICs. Besides the more traditional methods like Logistic Regression and Naive Bayes Classification we also discuss a different approach inspired by the statistical Monte Carlo methods. The numerical results are illustrated using ThreatGRID and ReversingLabs data. △ Less

Submitted 18 January, 2022; originally announced January 2022.

Comments: 10 pages, 2 figures

Diffusion Self-Organizing Map on the Hypersphere

Authors: M. Andrecut

Abstract: We discuss a diffusion based implementation of the self-organizing map on the unit hypersphere. We show that this approach can be efficiently implemented using just linear algebra methods, we give a python numpy implementation, and we illustrate the approach using the well known MNIST dataset. We discuss a diffusion based implementation of the self-organizing map on the unit hypersphere. We show that this approach can be efficiently implemented using just linear algebra methods, we give a python numpy implementation, and we illustrate the approach using the well known MNIST dataset. △ Less

Submitted 31 May, 2021; originally announced June 2021.

Comments: 10 pages, 4 figures

Additive Feature Hashing

Authors: M. Andrecut

Abstract: The hashing trick is a machine learning technique used to encode categorical features into a numerical vector representation of pre-defined fixed length. It works by using the categorical hash values as vector indices, and updating the vector values at those indices. Here we discuss a different approach based on additive-hashing and the "almost orthogonal" property of high-dimensional random vecto… ▽ More The hashing trick is a machine learning technique used to encode categorical features into a numerical vector representation of pre-defined fixed length. It works by using the categorical hash values as vector indices, and updating the vector values at those indices. Here we discuss a different approach based on additive-hashing and the "almost orthogonal" property of high-dimensional random vectors. That is, we show that additive feature hashing can be performed directly by adding the hash values and converting them into high-dimensional numerical vectors. We show that the performance of additive feature hashing is similar to the hashing trick, and we illustrate the results numerically using synthetic, language recognition, and SMS spam detection data. △ Less

Submitted 7 February, 2021; originally announced February 2021.

Comments: 11 pages 3 figures

K-Means Kernel Classifier

Authors: M. Andrecut

Abstract: We combine K-means clustering with the least-squares kernel classification method. K-means clustering is used to extract a set of representative vectors for each class. The least-squares kernel method uses these representative vectors as a training set for the classification task. We show that this combination of unsupervised and supervised learning algorithms performs very well, and we illustrate… ▽ More We combine K-means clustering with the least-squares kernel classification method. K-means clustering is used to extract a set of representative vectors for each class. The least-squares kernel method uses these representative vectors as a training set for the classification task. We show that this combination of unsupervised and supervised learning algorithms performs very well, and we illustrate this approach using the MNIST dataset △ Less

Submitted 23 December, 2020; originally announced December 2020.

Comments: 8 pages, 2 figures

High-Dimensional Vector Semantics

Authors: M. Andrecut

Abstract: In this paper we explore the "vector semantics" problem from the perspective of "almost orthogonal" property of high-dimensional random vectors. We show that this intriguing property can be used to "memorize" random vectors by simply adding them, and we provide an efficient probabilistic solution to the set membership problem. Also, we discuss several applications to word context vector embeddings… ▽ More In this paper we explore the "vector semantics" problem from the perspective of "almost orthogonal" property of high-dimensional random vectors. We show that this intriguing property can be used to "memorize" random vectors by simply adding them, and we provide an efficient probabilistic solution to the set membership problem. Also, we discuss several applications to word context vector embeddings, document sentences similarity, and spam filtering. △ Less

Submitted 23 February, 2018; originally announced February 2018.

Comments: 12 pages, 5 figures, Int. J. Mod. Phys. C, 2018

On the inherent competition between valid and spurious inductive inferences in Boolean data

Authors: M. Andrecut

Abstract: Inductive inference is the process of extracting general rules from specific observations. This problem also arises in the analysis of biological networks, such as genetic regulatory networks, where the interactions are complex and the observations are incomplete. A typical task in these problems is to extract general interaction rules as combinations of Boolean covariates, that explain a measured… ▽ More Inductive inference is the process of extracting general rules from specific observations. This problem also arises in the analysis of biological networks, such as genetic regulatory networks, where the interactions are complex and the observations are incomplete. A typical task in these problems is to extract general interaction rules as combinations of Boolean covariates, that explain a measured response variable. The inductive inference process can be considered as an incompletely specified Boolean function synthesis problem. This incompleteness of the problem will also generate spurious inferences, which are a serious threat to valid inductive inference rules. Using random Boolean data as a null model, here we attempt to measure the competition between valid and spurious inductive inference rules from a given data set. We formulate two greedy search algorithms, which synthesize a given Boolean response variable in a sparse disjunct normal form, and respectively a sparse generalized algebraic normal form of the variables from the observation data, and we evaluate numerically their performance. △ Less

Submitted 6 January, 2018; originally announced January 2018.

Comments: 12 pages, 2 figures, Int. J. Mod. Phys. C, 2017

Reservoir Computing on the Hypersphere

Authors: M. Andrecut

Abstract: Reservoir Computing (RC) refers to a Recurrent Neural Networks (RNNs) framework, frequently used for sequence learning and time series prediction. The RC system consists of a random fixed-weight RNN (the input-hidden reservoir layer) and a classifier (the hidden-output readout layer). Here we focus on the sequence learning problem, and we explore a different approach to RC. More specifically, we r… ▽ More Reservoir Computing (RC) refers to a Recurrent Neural Networks (RNNs) framework, frequently used for sequence learning and time series prediction. The RC system consists of a random fixed-weight RNN (the input-hidden reservoir layer) and a classifier (the hidden-output readout layer). Here we focus on the sequence learning problem, and we explore a different approach to RC. More specifically, we remove the non-linear neural activation function, and we consider an orthogonal reservoir acting on normalized states on the unit hypersphere. Surprisingly, our numerical results show that the system's memory capacity exceeds the dimensionality of the reservoir, which is the upper bound for the typical RC approach based on Echo State Networks (ESNs). We also show how the proposed system can be applied to symmetric cryptography problems, and we include a numerical implementation. △ Less

Submitted 23 June, 2017; originally announced June 2017.

Comments: 12 pages, 6 figures, Int. J. Mod. Phys. C, 2017

Randomized Kernel Methods for Least-Squares Support Vector Machines

Authors: M. Andrecut

Abstract: The least-squares support vector machine is a frequently used kernel method for non-linear regression and classification tasks. Here we discuss several approximation algorithms for the least-squares support vector machine classifier. The proposed methods are based on randomized block kernel matrices, and we show that they provide good accuracy and reliable scaling for multi-class classification pr… ▽ More The least-squares support vector machine is a frequently used kernel method for non-linear regression and classification tasks. Here we discuss several approximation algorithms for the least-squares support vector machine classifier. The proposed methods are based on randomized block kernel matrices, and we show that they provide good accuracy and reliable scaling for multi-class classification problems with relatively large data sets. Also, we present several numerical experiments that illustrate the practical applicability of the proposed methods. △ Less

Submitted 22 March, 2017; originally announced March 2017.

Comments: 16 pages, 6 figures

Journal ref: Int. J. Mod. Phys. C, 28, 1750015 (2017)

A Matrix Public Key Cryptosystem

Authors: M. Andrecut

Abstract: We discuss a matrix public key cryptosystem and its numerical implementation. We discuss a matrix public key cryptosystem and its numerical implementation. △ Less

Submitted 31 May, 2015; originally announced June 2015.

Comments: 18 pages, C code included

A String-Based Public Key Cryptosystem

Authors: M. Andrecut

Abstract: Traditional methods in public key cryptography are based on number theory, and suffer from problems such as dealing with very large numbers, making key creation cumbersome. Here, we propose a new public key cryptosystem based on strings only, which avoids the difficulties of the traditional number theory approach. The security mechanism for public and secret keys generation is ensured by a recursi… ▽ More Traditional methods in public key cryptography are based on number theory, and suffer from problems such as dealing with very large numbers, making key creation cumbersome. Here, we propose a new public key cryptosystem based on strings only, which avoids the difficulties of the traditional number theory approach. The security mechanism for public and secret keys generation is ensured by a recursive encoding mechanism embedded in a quasi-commutative-random function, resulted from the composition of a quasi-commutative function with a pseudo-random function. In this revised version of the paper we show that the eavesdropper's problem of the proposed cryptosystem has a solution, and we give the details of the solution. △ Less

Submitted 19 January, 2015; v1 submitted 5 September, 2014; originally announced October 2014.

Comments: In this revised version of the paper we show that the eavesdropper's problem of the proposed cryptosystem has a solution, and we give the details of the solution

ACM Class: E.3

Stochastic Recovery Of Sparse Signals From Random Measurements

Authors: M. Andrecut

Abstract: Sparse signal recovery from a small number of random measurements is a well known NP-hard to solve combinatorial optimization problem, with important applications in signal and image processing. The standard approach to the sparse signal recovery problem is based on the basis pursuit method. This approach requires the solution of a large convex optimization problem, and therefore suffers from high… ▽ More Sparse signal recovery from a small number of random measurements is a well known NP-hard to solve combinatorial optimization problem, with important applications in signal and image processing. The standard approach to the sparse signal recovery problem is based on the basis pursuit method. This approach requires the solution of a large convex optimization problem, and therefore suffers from high computational complexity. Here, we discuss a stochastic optimization method, as a low-complexity alternative to the basis pursuit approach. △ Less

Submitted 7 April, 2013; originally announced April 2013.

Comments: 6 pages, 3 figures

Journal ref: Engineering Letters, 19:1, EL_19_1_01 (2010)

Sparse Random Approximation and Lossy Compression

Authors: M. Andrecut

Abstract: We discuss a method for sparse signal approximation, which is based on the correlation of the target signal with a pseudo-random signal, and uses a modification of the greedy matching pursuit algorithm. We show that this approach provides an efficient encoding-decoding method, which can be used also for lossy compression and encryption purposes. We discuss a method for sparse signal approximation, which is based on the correlation of the target signal with a pseudo-random signal, and uses a modification of the greedy matching pursuit algorithm. We show that this approach provides an efficient encoding-decoding method, which can be used also for lossy compression and encryption purposes. △ Less

Submitted 25 May, 2011; originally announced May 2011.

Comments: 6 pages, 4 figures

Parallel GPU Implementation of Iterative PCA Algorithms

Authors: M. Andrecut

Abstract: Principal component analysis (PCA) is a key statistical technique for multivariate data analysis. For large data sets the common approach to PCA computation is based on the standard NIPALS-PCA algorithm, which unfortunately suffers from loss of orthogonality, and therefore its applicability is usually limited to the estimation of the first few components. Here we present an algorithm based on Gr… ▽ More Principal component analysis (PCA) is a key statistical technique for multivariate data analysis. For large data sets the common approach to PCA computation is based on the standard NIPALS-PCA algorithm, which unfortunately suffers from loss of orthogonality, and therefore its applicability is usually limited to the estimation of the first few components. Here we present an algorithm based on Gram-Schmidt orthogonalization (called GS-PCA), which eliminates this shortcoming of NIPALS-PCA. Also, we discuss the GPU (Graphics Processing Unit) parallel implementation of both NIPALS-PCA and GS-PCA algorithms. The numerical results show that the GPU parallel optimized versions, based on CUBLAS (NVIDIA) are substantially faster (up to 12 times) than the CPU optimized versions based on CBLAS (GNU Scientific Library). △ Less

Submitted 6 November, 2008; originally announced November 2008.

Comments: 45 pages, 1 figure, source code included