-
Using evolutionary computation to optimize task performance of unclocked, recurrent Boolean circuits in FPGAs
Authors:
Raphael Norman-Tenazas,
David Kleinberg,
Erik C. Johnson,
Daniel P. Lathrop,
Matthew J. Roos
Abstract:
It has been shown that unclocked, recurrent networks of Boolean gates in FPGAs can be used for low-SWaP reservoir computing. In such systems, topology and node functionality of the network are randomly initialized. To create a network that solves a task, weights are applied to output nodes and learning is achieved by adjusting those weights with conventional machine learning methods. However, perf…
▽ More
It has been shown that unclocked, recurrent networks of Boolean gates in FPGAs can be used for low-SWaP reservoir computing. In such systems, topology and node functionality of the network are randomly initialized. To create a network that solves a task, weights are applied to output nodes and learning is achieved by adjusting those weights with conventional machine learning methods. However, performance is often limited compared to networks where all parameters are learned. Herein, we explore an alternative learning approach for unclocked, recurrent networks in FPGAs. We use evolutionary computation to evolve the Boolean functions of network nodes. In one type of implementation the output nodes are used directly to perform a task and all learning is via evolution of the network's node functions. In a second type of implementation a back-end classifier is used as in traditional reservoir computing. In that case, both evolution of node functions and adjustment of output node weights contribute to learning. We demonstrate the practicality of node function evolution, obtaining an accuracy improvement of ~30% on an image classification task while processing at a rate of over three million samples per second. We additionally demonstrate evolvability of network memory and dynamic output signals.
△ Less
Submitted 19 March, 2024;
originally announced March 2024.
-
Exploiting Large Neuroimaging Datasets to Create Connectome-Constrained Approaches for more Robust, Efficient, and Adaptable Artificial Intelligence
Authors:
Erik C. Johnson,
Brian S. Robinson,
Gautam K. Vallabha,
Justin Joyce,
Jordan K. Matelsky,
Raphael Norman-Tenazas,
Isaac Western,
Marisel Villafañe-Delgado,
Martha Cervantes,
Michael S. Robinette,
Arun V. Reddy,
Lindsey Kitchell,
Patricia K. Rivlin,
Elizabeth P. Reilly,
Nathan Drenkow,
Matthew J. Roos,
I-Jeng Wang,
Brock A. Wester,
William R. Gray-Roncal,
Joan A. Hoffmann
Abstract:
Despite the progress in deep learning networks, efficient learning at the edge (enabling adaptable, low-complexity machine learning solutions) remains a critical need for defense and commercial applications. We envision a pipeline to utilize large neuroimaging datasets, including maps of the brain which capture neuron and synapse connectivity, to improve machine learning approaches. We have pursue…
▽ More
Despite the progress in deep learning networks, efficient learning at the edge (enabling adaptable, low-complexity machine learning solutions) remains a critical need for defense and commercial applications. We envision a pipeline to utilize large neuroimaging datasets, including maps of the brain which capture neuron and synapse connectivity, to improve machine learning approaches. We have pursued different approaches within this pipeline structure. First, as a demonstration of data-driven discovery, the team has developed a technique for discovery of repeated subcircuits, or motifs. These were incorporated into a neural architecture search approach to evolve network architectures. Second, we have conducted analysis of the heading direction circuit in the fruit fly, which performs fusion of visual and angular velocity features, to explore augmenting existing computational models with new insight. Our team discovered a novel pattern of connectivity, implemented a new model, and demonstrated sensor fusion on a robotic platform. Third, the team analyzed circuitry for memory formation in the fruit fly connectome, enabling the design of a novel generative replay approach. Finally, the team has begun analysis of connectivity in mammalian cortex to explore potential improvements to transformer networks. These constraints increased network robustness on the most challenging examples in the CIFAR-10-C computer vision robustness benchmark task, while reducing learnable attention parameters by over an order of magnitude. Taken together, these results demonstrate multiple potential approaches to utilize insight from neural systems for developing robust and efficient machine learning techniques.
△ Less
Submitted 26 May, 2023;
originally announced May 2023.
-
Towards Robust Image-in-Audio Deep Steganography
Authors:
Jaume Ros,
Margarita Geleta,
Jordi Pons,
Xavier Giro-i-Nieto
Abstract:
The field of steganography has experienced a surge of interest due to the recent advancements in AI-powered techniques, particularly in the context of multimodal setups that enable the concealment of signals within signals of a different nature. The primary objectives of all steganographic methods are to achieve perceptual transparency, robustness, and large embedding capacity - which often presen…
▽ More
The field of steganography has experienced a surge of interest due to the recent advancements in AI-powered techniques, particularly in the context of multimodal setups that enable the concealment of signals within signals of a different nature. The primary objectives of all steganographic methods are to achieve perceptual transparency, robustness, and large embedding capacity - which often present conflicting goals that classical methods have struggled to reconcile. This paper extends and enhances an existing image-in-audio deep steganography method by focusing on improving its robustness. The proposed enhancements include modifications to the loss function, utilization of the Short-Time Fourier Transform (STFT), introduction of redundancy in the encoding process for error correction, and buffering of additional information in the pixel subconvolution operation. The results demonstrate that our approach outperforms the existing method in terms of robustness and perceptual transparency.
△ Less
Submitted 14 March, 2023; v1 submitted 8 March, 2023;
originally announced March 2023.
-
Estimation of lateral track irregularity through Kalman filtering techniques
Authors:
S. Munoz,
J. Ros,
J. L. Escalona
Abstract:
The aim of this work is to develop a model-based methodology for monitoring lateral track irregularities based on the use of inertial sensors mounted on an in-service train. To this end, a gyroscope is used to measure the wheelset yaw angular velocity and two accelerometers are used to measure lateral acceleration of the wheelset and the bogie frame. Using a highly simplified linear bogie model th…
▽ More
The aim of this work is to develop a model-based methodology for monitoring lateral track irregularities based on the use of inertial sensors mounted on an in-service train. To this end, a gyroscope is used to measure the wheelset yaw angular velocity and two accelerometers are used to measure lateral acceleration of the wheelset and the bogie frame. Using a highly simplified linear bogie model that is able to capture the most relevant dynamic behaviour allows for the set-up of a very efficient Kalman-based monitoring strategy. The behaviour of the designed filter is assessed through the use of a detailed multibody model of an in-service vehicle running on a straight track with realistic irregularities. The model output is used to generate virtual measurements that are subsequently used to run the filter and validate the proposed estimator. In addition, the equivalent parameters of the simplified model are identified based on these simulations. In order to prove the robustness of the proposed technique, a systematic parametric analysis has been performed. The results obtained with the proposed method are promising, showing high accuracy and robustness for monitoring lateral alignment on straight tracks, with a very low computational cost.
△ Less
Submitted 11 March, 2020;
originally announced March 2020.
-
Utilizing a null class to restrict decision spaces and defend against neural network adversarial attacks
Authors:
Matthew J. Roos
Abstract:
Despite recent progress, deep neural networks generally continue to be vulnerable to so-called adversarial examples--input images with small perturbations that can result in changes in the output classifications, despite no such change in the semantic meaning to human viewers. This is true even for seemingly simple challenges such as the MNIST digit classification task. In part, this suggests that…
▽ More
Despite recent progress, deep neural networks generally continue to be vulnerable to so-called adversarial examples--input images with small perturbations that can result in changes in the output classifications, despite no such change in the semantic meaning to human viewers. This is true even for seemingly simple challenges such as the MNIST digit classification task. In part, this suggests that these networks are not relying on the same set of object features as humans use to make these classifications. In this paper we examine an additional, and largely unexplored, cause behind this phenomenon--namely, the use of the conventional training paradigm in which the entire input space is parcellated among the training classes. Owing to this paradigm, learned decision spaces for individual classes span excessively large regions of the input space and include images that have no semantic similarity to images in the training set. In this study, we train models that include a null class. That is, models may "opt-out" of classifying an input image as one of the digit classes. During training, null images are created through a variety of methods, in an attempt to create tighter and more semantically meaningful decision spaces for the digit classes. The best performing models classify nearly all adversarial examples as nulls, rather than mistaking them as a member of an incorrect digit class, while simultaneously maintaining high accuracy on the unperturbed test set. The use of a null class and the training paradigm presented herein may provide an effective defense against adversarial attacks for some applications. Code for replicating this study will be made available at https://github.com/mattroos/null_class_adversarial_defense .
△ Less
Submitted 24 February, 2020;
originally announced February 2020.
-
Drawbacks and alternatives to the numerical calculation of the base inertial parameters expressions for low mobility mechanisms
Authors:
Xabier Iriarte,
Javier Ros,
Aitor Plaza,
Jokin Aginaga,
Vicente Mata
Abstract:
Base inertial parameters constitute a minimal inertial parametrization of mechanical systems that is of interest, for example, in parameter estimation and model reduction. Numerical and symbolic methods are available to determine their expressions. In this paper the problems associated with the numerical determination of the base inertial parameters expressions in the context of low mobility mecha…
▽ More
Base inertial parameters constitute a minimal inertial parametrization of mechanical systems that is of interest, for example, in parameter estimation and model reduction. Numerical and symbolic methods are available to determine their expressions. In this paper the problems associated with the numerical determination of the base inertial parameters expressions in the context of low mobility mechanisms are analyzed and discussed through and example. To circumvent these problems two alternatives are proposed: a variable precision arithmetic implementation of the customary numerical algorithm and the application of a general symbolic method. Finally, the advantages of both approaches compared to the numerical one are discussed in the context of the proposed low mobility example.
△ Less
Submitted 28 June, 2017;
originally announced June 2017.
-
Symbolic Multibody Methods for Real-Time Simulation of Railway Vehicles
Authors:
Javier Ros,
Aitor Plaza,
Xabier Iriarte,
Jesús María Pintor
Abstract:
In this work, recently developed state-of-the-art symbolic multibody methods are tested to accurately model a complex railway vehicle. The model is generated using a symbolic implementation of the principle of the virtual power. Creep forces are modeled using a direct symbolic implementation of the standard linear Kalker model. No simplifications, as base parameter reduction, partial-linearization…
▽ More
In this work, recently developed state-of-the-art symbolic multibody methods are tested to accurately model a complex railway vehicle. The model is generated using a symbolic implementation of the principle of the virtual power. Creep forces are modeled using a direct symbolic implementation of the standard linear Kalker model. No simplifications, as base parameter reduction, partial-linearization or look-up tables for contact kinematics, are used. An Implicit-Explicit integration scheme is proposed to efficiently deal with the stiff creep dynamics. Hard real-time performance is achieved: the CPU time required for a very stable 1 ms integration time step is 256 μs.
△ Less
Submitted 6 June, 2017;
originally announced June 2017.
-
Simplification of multibody models by parameter reduction
Authors:
Javier Ros,
Xabier Iriarte,
Aitor Plaza,
Vicente Mata
Abstract:
Model selection methods are used in different scientific contexts to represent a characteristic data set in terms of a reduced number of parameters. Apparently, these methods have not found their way into the literature on multibody systems dynamics. Multibody models can be considered parametric models in terms of their dynamic parameters, and model selection techniques can then be used to express…
▽ More
Model selection methods are used in different scientific contexts to represent a characteristic data set in terms of a reduced number of parameters. Apparently, these methods have not found their way into the literature on multibody systems dynamics. Multibody models can be considered parametric models in terms of their dynamic parameters, and model selection techniques can then be used to express these models in terms of a reduced number of parameters. These parameter-reduced models are expected to have a smaller computational complexity than the original one and still preserve the desired level of accuracy. They are also known to be good candidates for parameter estimation purposes.
In this work, simulations of the actual model are used to define a data set that is representative of the system's standard working conditions. A parameter-reduced model is chosen and its parameter values estimated so that they minimize the prediction error on these data. To that end, model selection heuristics and normalized error measures are proposed.
Using this methodology, two multibody systems with very different characteristic mobility are analyzed. Highly considerable reductions in the number of parameters and computational cost are obtained without compromising the accuracy of the reduced model too much. As an additional result, a generalization of the base parameter concept to the context of parameter-reduced models is proposed.
△ Less
Submitted 29 May, 2017;
originally announced May 2017.
