Showing 1–38 of 38 results for author: Alexandrov, B S

Breaking the Curse of Dimensionality: Solving Configurational Integrals for Crystalline Solids by Tensor Networks

Authors: Duc P. Truong, Benjamin Nebgen, Derek DeSantis, Dimiter N. Petsev, Kim Ø. Rasmussen, Boian S. Alexandrov

Abstract: Accurately evaluating configurational integrals for dense solids remains a central and difficult challenge in the statistical mechanics of condensed systems. Here, we present a novel tensor network approach that reformulates the high-dimensional configurational integral for identical-particle crystals into a sequence of computationally efficient summations. We represent the integrand as a high-dim… ▽ More Accurately evaluating configurational integrals for dense solids remains a central and difficult challenge in the statistical mechanics of condensed systems. Here, we present a novel tensor network approach that reformulates the high-dimensional configurational integral for identical-particle crystals into a sequence of computationally efficient summations. We represent the integrand as a high-dimensional tensor and apply tensor-train (TT) decomposition together with a custom TT-cross interpolation scheme. This approach avoids the need to explicitly construct the full tensor, which would otherwise be computationally intractable. We introduce tailored rank-1 and rank-2 schemes optimized for sharply peaked Boltzmann probability densities, typical in crystalline solids. When applied to the calculation of internal energy and pressure-temperature curves for crystalline copper (Cu) and argon (Ar), as well as the alpha-to-beta phase transition in tin (Sn), our method accurately reproduces molecular dynamics simulation results using tight-binding, machine learning (HIP-NN), and MEAM potentials, all within seconds of computation time. △ Less

Submitted 27 May, 2025; originally announced May 2025.

Bridging Legal Knowledge and AI: Retrieval-Augmented Generation with Vector Stores, Knowledge Graphs, and Hierarchical Non-negative Matrix Factorization

Authors: Ryan C. Barron, Maksim E. Eren, Olga M. Serafimova, Cynthia Matuszek, Boian S. Alexandrov

Abstract: Agentic Generative AI, powered by Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG), Knowledge Graphs (KGs), and Vector Stores (VSs), represents a transformative technology applicable to specialized domains such as legal systems, research, recommender systems, cybersecurity, and global security, including proliferation research. This technology excels at inferring relationship… ▽ More Agentic Generative AI, powered by Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG), Knowledge Graphs (KGs), and Vector Stores (VSs), represents a transformative technology applicable to specialized domains such as legal systems, research, recommender systems, cybersecurity, and global security, including proliferation research. This technology excels at inferring relationships within vast unstructured or semi-structured datasets. The legal domain here comprises complex data characterized by extensive, interrelated, and semi-structured knowledge systems with complex relations. It comprises constitutions, statutes, regulations, and case law. Extracting insights and navigating the intricate networks of legal documents and their relations is crucial for effective legal research. Here, we introduce a generative AI system that integrates RAG, VS, and KG, constructed via Non-Negative Matrix Factorization (NMF), to enhance legal information retrieval and AI reasoning and minimize hallucinations. In the legal system, these technologies empower AI agents to identify and analyze complex connections among cases, statutes, and legal precedents, uncovering hidden relationships and predicting legal trends-challenging tasks that are essential for ensuring justice and improving operational efficiency. Our system employs web scraping techniques to systematically collect legal texts, such as statutes, constitutional provisions, and case law, from publicly accessible platforms like Justia. It bridges the gap between traditional keyword-based searches and contextual understanding by leveraging advanced semantic representations, hierarchical relationships, and latent topic discovery. This framework supports legal document clustering, summarization, and cross-referencing, for scalable, interpretable, and accurate retrieval for semi-structured data while advancing computational law and AI. △ Less

Submitted 8 May, 2025; v1 submitted 27 February, 2025; originally announced February 2025.

Comments: 10 pages, 8 figures, 5 tables

Space-Time Spectral Element Tensor Network Approach for Time Dependent Convection Diffusion Reaction Equation with Variable Coefficients

Authors: Dibyendu Adak, Duc P. Truong, Radoslav Vuchkov, Saibal De, Derek DeSantis, Nathan V. Roberts, Kim Ø. Rasmussen, Boian S. Alexandrov

Abstract: In this paper, we present a new space-time Petrov-Galerkin-like method. This method utilizes a mixed formulation of Tensor Train (TT) and Quantized Tensor Train (QTT), designed for the spectral element discretization (Q1-SEM) of the time-dependent convection-diffusion-reaction (CDR) equation. We reformulate the assembly process of the spectral element discretized CDR to enhance its compatibility w… ▽ More In this paper, we present a new space-time Petrov-Galerkin-like method. This method utilizes a mixed formulation of Tensor Train (TT) and Quantized Tensor Train (QTT), designed for the spectral element discretization (Q1-SEM) of the time-dependent convection-diffusion-reaction (CDR) equation. We reformulate the assembly process of the spectral element discretized CDR to enhance its compatibility with tensor operations and introduce a low-rank tensor structure for the spectral element operators. Recognizing the banded structure inherent in the spectral element framework's discrete operators, we further exploit the QTT format of the CDR to achieve greater speed and compression. Additionally, we present a comprehensive approach for integrating variable coefficients of CDR into the global discrete operators within the TT/QTT framework. The effectiveness of the proposed method, in terms of memory efficiency and computational complexity, is demonstrated through a series of numerical experiments, including a semi-linear example. △ Less

Submitted 6 November, 2024; originally announced November 2024.

Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization

Authors: Ryan C. Barron, Ves Grantcharov, Selma Wanna, Maksim E. Eren, Manish Bhattarai, Nicholas Solovyev, George Tompkins, Charles Nicholas, Kim Ø. Rasmussen, Cynthia Matuszek, Boian S. Alexandrov

Abstract: Large Language Models (LLMs) are pre-trained on large-scale corpora and excel in numerous general natural language processing (NLP) tasks, such as question answering (QA). Despite their advanced language capabilities, when it comes to domain-specific and knowledge-intensive tasks, LLMs suffer from hallucinations, knowledge cut-offs, and lack of knowledge attributions. Additionally, fine tuning LLM… ▽ More Large Language Models (LLMs) are pre-trained on large-scale corpora and excel in numerous general natural language processing (NLP) tasks, such as question answering (QA). Despite their advanced language capabilities, when it comes to domain-specific and knowledge-intensive tasks, LLMs suffer from hallucinations, knowledge cut-offs, and lack of knowledge attributions. Additionally, fine tuning LLMs' intrinsic knowledge to highly specific domains is an expensive and time consuming process. The retrieval-augmented generation (RAG) process has recently emerged as a method capable of optimization of LLM responses, by referencing them to a predetermined ontology. It was shown that using a Knowledge Graph (KG) ontology for RAG improves the QA accuracy, by taking into account relevant sub-graphs that preserve the information in a structured manner. In this paper, we introduce SMART-SLIC, a highly domain-specific LLM framework, that integrates RAG with KG and a vector store (VS) that store factual domain specific information. Importantly, to avoid hallucinations in the KG, we build these highly domain-specific KGs and VSs without the use of LLMs, but via NLP, data mining, and nonnegative tensor factorization with automatic model selection. Pairing our RAG with a domain-specific: (i) KG (containing structured information), and (ii) VS (containing unstructured information) enables the development of domain-specific chat-bots that attribute the source of information, mitigate hallucinations, lessen the need for fine-tuning, and excel in highly domain-specific question answering tasks. We pair SMART-SLIC with chain-of-thought prompting agents. The framework is designed to be generalizable to adapt to any specific or specialized domain. In this paper, we demonstrate the question answering capabilities of our framework on a corpus of scientific publications on malware analysis and anomaly detection. △ Less

Submitted 3 October, 2024; originally announced October 2024.

Comments: 9 pages 7 figures, 1 table, 1 cypher code Accepted to ICMLA 2024

High-order Tensor-Train Finite Volume Method for Shallow Water Equations

Authors: Mustafa Engin Danis, Duc P. Truong, Derek DeSantis, Mark Petersen, Kim O. Rasmussen, Boian S. Alexandrov

Abstract: In this paper, we introduce a high-order tensor-train (TT) finite volume method for the Shallow Water Equations (SWEs). We present the implementation of the $3^{rd}$ order Upwind and the $5^{th}$ order Upwind and WENO reconstruction schemes in the TT format. It is shown in detail that the linear upwind schemes can be implemented by directly manipulating the TT cores while the WENO scheme requires… ▽ More In this paper, we introduce a high-order tensor-train (TT) finite volume method for the Shallow Water Equations (SWEs). We present the implementation of the $3^{rd}$ order Upwind and the $5^{th}$ order Upwind and WENO reconstruction schemes in the TT format. It is shown in detail that the linear upwind schemes can be implemented by directly manipulating the TT cores while the WENO scheme requires the use of TT cross interpolation for the nonlinear reconstruction. In the development of numerical fluxes, we directly compute the flux for the linear SWEs without using TT rounding or cross interpolation. For the nonlinear SWEs where the TT reciprocal of the shallow water layer thickness is needed for fluxes, we develop an approximation algorithm using Taylor series to compute the TT reciprocal. The performance of the TT finite volume solver with linear and nonlinear reconstruction options is investigated under a physically relevant set of validation problems. In all test cases, the TT finite volume method maintains the formal high-order accuracy of the corresponding traditional finite volume method. In terms of speed, the TT solver achieves up to 124x acceleration of the traditional full-tensor scheme. △ Less

Submitted 6 August, 2024; originally announced August 2024.

Report number: LA-UR-24-28480 MSC Class: 65M06; 86A08

TopicTag: Automatic Annotation of NMF Topic Models Using Chain of Thought and Prompt Tuning with LLMs

Authors: Selma Wanna, Ryan Barron, Nick Solovyev, Maksim E. Eren, Manish Bhattarai, Kim Rasmussen, Boian S. Alexandrov

Abstract: Topic modeling is a technique for organizing and extracting themes from large collections of unstructured text. Non-negative matrix factorization (NMF) is a common unsupervised approach that decomposes a term frequency-inverse document frequency (TF-IDF) matrix to uncover latent topics and segment the dataset accordingly. While useful for highlighting patterns and clustering documents, NMF does no… ▽ More Topic modeling is a technique for organizing and extracting themes from large collections of unstructured text. Non-negative matrix factorization (NMF) is a common unsupervised approach that decomposes a term frequency-inverse document frequency (TF-IDF) matrix to uncover latent topics and segment the dataset accordingly. While useful for highlighting patterns and clustering documents, NMF does not provide explicit topic labels, necessitating subject matter experts (SMEs) to assign labels manually. We present a methodology for automating topic labeling in documents clustered via NMF with automatic model determination (NMFk). By leveraging the output of NMFk and employing prompt engineering, we utilize large language models (LLMs) to generate accurate topic labels. Our case study on over 34,000 scientific abstracts on Knowledge Graphs demonstrates the effectiveness of our method in enhancing knowledge management and document organization. △ Less

Submitted 28 July, 2024; originally announced July 2024.

Comments: Accepted to ACM Symposium on Document Engineering 2024 (DocEng 24), 2024

Binary Bleed: Fast Distributed and Parallel Method for Automatic Model Selection

Authors: Ryan Barron, Maksim E. Eren, Manish Bhattarai, Ismael Boureima, Cynthia Matuszek, Boian S. Alexandrov

Abstract: In several Machine Learning (ML) clustering and dimensionality reduction approaches, such as non-negative matrix factorization (NMF), RESCAL, and K-Means clustering, users must select a hyper-parameter k to define the number of clusters or components that yield an ideal separation of samples or clean clusters. This selection, while difficult, is crucial to avoid overfitting or underfitting the dat… ▽ More In several Machine Learning (ML) clustering and dimensionality reduction approaches, such as non-negative matrix factorization (NMF), RESCAL, and K-Means clustering, users must select a hyper-parameter k to define the number of clusters or components that yield an ideal separation of samples or clean clusters. This selection, while difficult, is crucial to avoid overfitting or underfitting the data. Several ML applications use scoring methods (e.g., Silhouette and Davies Boulding scores) to evaluate the cluster pattern stability for a specific k. The score is calculated for different trials over a range of k, and the ideal k is heuristically selected as the value before the model starts overfitting, indicated by a drop or increase in the score resembling an elbow curve plot. While the grid-search method can be used to accurately find a good k value, visiting a range of k can become time-consuming and computationally resource-intensive. In this paper, we introduce the Binary Bleed method based on binary search, which significantly reduces the k search space for these grid-search ML algorithms by truncating the target k values from the search space using a heuristic with thresholding over the scores. Binary Bleed is designed to work with single-node serial, single-node multi-processing, and distributed computing resources. In our experiments, we demonstrate the reduced search space gain over a naive sequential search of the ideal k and the accuracy of the Binary Bleed in identifying the correct k for NMFk, K-Means pyDNMFk, and pyDRESCALk with Silhouette and Davies Boulding scores. We make our implementation of Binary Bleed for the NMF algorithm available on GitHub. △ Less

Submitted 26 July, 2024; originally announced July 2024.

Comments: 8 pages, submitted to IEEE HPEC

Tensor Network Space-Time Spectral Collocation Method for Solving the Nonlinear Convection Diffusion Equation

Authors: Dibyendu Adak, M. Engin Danis, Duc P. Truong, Kim Ø. Rasmussen, Boian S. Alexandrov

Abstract: Spectral methods provide highly accurate numerical solutions for partial differential equations, exhibiting exponential convergence with the number of spectral nodes. Traditionally, in addressing time-dependent nonlinear problems, attention has been on low-order finite difference schemes for time discretization and spectral element schemes for spatial variables. However, our recent developments ha… ▽ More Spectral methods provide highly accurate numerical solutions for partial differential equations, exhibiting exponential convergence with the number of spectral nodes. Traditionally, in addressing time-dependent nonlinear problems, attention has been on low-order finite difference schemes for time discretization and spectral element schemes for spatial variables. However, our recent developments have resulted in the application of spectral methods to both space and time variables, preserving spectral convergence in both domains. Leveraging Tensor Train techniques, our approach tackles the curse of dimensionality inherent in space-time methods. Here, we extend this methodology to the nonlinear time-dependent convection-diffusion equation. Our discretization scheme exhibits a low-rank structure, facilitating translation to tensor-train (TT) format. Nevertheless, controlling the TT-rank across Newton's iterations, needed to deal with the nonlinearity, poses a challenge, leading us to devise the "Step Truncation TT-Newton" method. We demonstrate the exponential convergence of our methods through various benchmark examples. Importantly, our scheme offers significantly reduced memory requirement compared to the full-grid scheme. △ Less

Submitted 4 June, 2024; originally announced June 2024.

MSC Class: 65N12; 65N25; 15A23; 15A69

Cyber-Security Knowledge Graph Generation by Hierarchical Nonnegative Matrix Factorization

Authors: Ryan Barron, Maksim E. Eren, Manish Bhattarai, Selma Wanna, Nicholas Solovyev, Kim Rasmussen, Boian S. Alexandrov, Charles Nicholas, Cynthia Matuszek

Abstract: Much of human knowledge in cybersecurity is encapsulated within the ever-growing volume of scientific papers. As this textual data continues to expand, the importance of document organization methods becomes increasingly crucial for extracting actionable insights hidden within large text datasets. Knowledge Graphs (KGs) serve as a means to store factual information in a structured manner, providin… ▽ More Much of human knowledge in cybersecurity is encapsulated within the ever-growing volume of scientific papers. As this textual data continues to expand, the importance of document organization methods becomes increasingly crucial for extracting actionable insights hidden within large text datasets. Knowledge Graphs (KGs) serve as a means to store factual information in a structured manner, providing explicit, interpretable knowledge that includes domain-specific information from the cybersecurity scientific literature. One of the challenges in constructing a KG from scientific literature is the extraction of ontology from unstructured text. In this paper, we address this topic and introduce a method for building a multi-modal KG by extracting structured ontology from scientific papers. We demonstrate this concept in the cybersecurity domain. One modality of the KG represents observable information from the papers, such as the categories in which they were published or the authors. The second modality uncovers latent (hidden) patterns of text extracted through hierarchical and semantic non-negative matrix factorization (NMF), such as named entities, topics or clusters, and keywords. We illustrate this concept by consolidating more than two million scientific papers uploaded to arXiv into the cyber-domain, using hierarchical and semantic NMF, and by building a cyber-domain-specific KG. △ Less

Submitted 26 March, 2024; v1 submitted 24 March, 2024; originally announced March 2024.

Comments: Accepted at IEEE ISDFS

Catch'em all: Classification of Rare, Prominent, and Novel Malware Families

Authors: Maksim E. Eren, Ryan Barron, Manish Bhattarai, Selma Wanna, Nicholas Solovyev, Kim Rasmussen, Boian S. Alexandrov, Charles Nicholas

Abstract: National security is threatened by malware, which remains one of the most dangerous and costly cyber threats. As of last year, researchers reported 1.3 billion known malware specimens, motivating the use of data-driven machine learning (ML) methods for analysis. However, shortcomings in existing ML approaches hinder their mass adoption. These challenges include detection of novel malware and the a… ▽ More National security is threatened by malware, which remains one of the most dangerous and costly cyber threats. As of last year, researchers reported 1.3 billion known malware specimens, motivating the use of data-driven machine learning (ML) methods for analysis. However, shortcomings in existing ML approaches hinder their mass adoption. These challenges include detection of novel malware and the ability to perform malware classification in the face of class imbalance: a situation where malware families are not equally represented in the data. Our work addresses these shortcomings with MalwareDNA: an advanced dimensionality reduction and feature extraction framework. We demonstrate stable task performance under class imbalance for the following tasks: malware family classification and novel malware detection with a trade-off in increased abstention or reject-option rate. △ Less

Submitted 4 March, 2024; originally announced March 2024.

Comments: Accepted at IEEE 12th International Symposium on Digital Forensics and Security (ISDFS), 2024. arXiv admin note: text overlap with arXiv:2309.01350

Tensor Network Space-Time Spectral Collocation Method for Time Dependent Convection-Diffusion-Reaction Equations

Authors: Dibyendu Adak, Duc P. Truong, Gianmarco Manzini, Kim Ø. Rasmussen, Boian S. Alexandrov

Abstract: Emerging tensor network techniques for solutions of Partial Differential Equations (PDEs), known for their ability to break the curse of dimensionality, deliver new mathematical methods for ultrafast numerical solutions of high-dimensional problems. Here, we introduce a Tensor Train (TT) Chebyshev spectral collocation method, in both space and time, for solution of the time dependent convection-di… ▽ More Emerging tensor network techniques for solutions of Partial Differential Equations (PDEs), known for their ability to break the curse of dimensionality, deliver new mathematical methods for ultrafast numerical solutions of high-dimensional problems. Here, we introduce a Tensor Train (TT) Chebyshev spectral collocation method, in both space and time, for solution of the time dependent convection-diffusion-reaction (CDR) equation with inhomogeneous boundary conditions, in Cartesian geometry. Previous methods for numerical solution of time dependent PDEs often use finite difference for time, and a spectral scheme for the spatial dimensions, which leads to slow linear convergence. Spectral collocation space-time methods show exponential convergence, however, for realistic problems they need to solve large four-dimensional systems. We overcome this difficulty by using a TT approach as its complexity only grows linearly with the number of dimensions. We show that our TT space-time Chebyshev spectral collocation method converges exponentially, when the solution of the CDR is smooth, and demonstrate that it leads to very high compression of linear operators from terabytes to kilobytes in TT-format, and tens of thousands times speedup when compared to full grid space-time spectral method. These advantages allow us to obtain the solutions at much higher resolutions. △ Less

Submitted 28 February, 2024; originally announced February 2024.

MSC Class: 15A69; 35Q79; 65M70

Interactive Distillation of Large Single-Topic Corpora of Scientific Papers

Authors: Nicholas Solovyev, Ryan Barron, Manish Bhattarai, Maksim E. Eren, Kim O. Rasmussen, Boian S. Alexandrov

Abstract: Highly specific datasets of scientific literature are important for both research and education. However, it is difficult to build such datasets at scale. A common approach is to build these datasets reductively by applying topic modeling on an established corpus and selecting specific topics. A more robust but time-consuming approach is to build the dataset constructively in which a subject matte… ▽ More Highly specific datasets of scientific literature are important for both research and education. However, it is difficult to build such datasets at scale. A common approach is to build these datasets reductively by applying topic modeling on an established corpus and selecting specific topics. A more robust but time-consuming approach is to build the dataset constructively in which a subject matter expert (SME) handpicks documents. This method does not scale and is prone to error as the dataset grows. Here we showcase a new tool, based on machine learning, for constructively generating targeted datasets of scientific literature. Given a small initial "core" corpus of papers, we build a citation network of documents. At each step of the citation network, we generate text embeddings and visualize the embeddings through dimensionality reduction. Papers are kept in the dataset if they are "similar" to the core or are otherwise pruned through human-in-the-loop selection. Additional insight into the papers is gained through sub-topic modeling using SeNMFk. We demonstrate our new tool for literature review by applying it to two different fields in machine learning. △ Less

Submitted 19 September, 2023; originally announced September 2023.

Comments: Accepted at 2023 IEEE ICMLA conference

Semi-supervised Classification of Malware Families Under Extreme Class Imbalance via Hierarchical Non-Negative Matrix Factorization with Automatic Model Selection

Authors: Maksim E. Eren, Manish Bhattarai, Robert J. Joyce, Edward Raff, Charles Nicholas, Boian S. Alexandrov

Abstract: Identification of the family to which a malware specimen belongs is essential in understanding the behavior of the malware and developing mitigation strategies. Solutions proposed by prior work, however, are often not practicable due to the lack of realistic evaluation factors. These factors include learning under class imbalance, the ability to identify new malware, and the cost of production-qua… ▽ More Identification of the family to which a malware specimen belongs is essential in understanding the behavior of the malware and developing mitigation strategies. Solutions proposed by prior work, however, are often not practicable due to the lack of realistic evaluation factors. These factors include learning under class imbalance, the ability to identify new malware, and the cost of production-quality labeled data. In practice, deployed models face prominent, rare, and new malware families. At the same time, obtaining a large quantity of up-to-date labeled malware for training a model can be expensive. In this paper, we address these problems and propose a novel hierarchical semi-supervised algorithm, which we call the HNMFk Classifier, that can be used in the early stages of the malware family labeling process. Our method is based on non-negative matrix factorization with automatic model selection, that is, with an estimation of the number of clusters. With HNMFk Classifier, we exploit the hierarchical structure of the malware data together with a semi-supervised setup, which enables us to classify malware families under conditions of extreme class imbalance. Our solution can perform abstaining predictions, or rejection option, which yields promising results in the identification of novel malware families and helps with maintaining the performance of the model when a low quantity of labeled data is used. We perform bulk classification of nearly 2,900 both rare and prominent malware families, through static analysis, using nearly 388,000 samples from the EMBER-2018 corpus. In our experiments, we surpass both supervised and semi-supervised baseline models with an F1 score of 0.80. △ Less

Submitted 12 September, 2023; originally announced September 2023.

Comments: Accepted at ACM TOPS

Tensor Networks for Solving Realistic Time-independent Boltzmann Neutron Transport Equation

Authors: Duc P. Truong, Mario I. Ortega, Ismael Boureima, Gianmarco Manzini, Kim Ø. Rasmussen, Boian S. Alexandrov

Abstract: Tensor network techniques, known for their low-rank approximation ability that breaks the curse of dimensionality, are emerging as a foundation of new mathematical methods for ultra-fast numerical solutions of high-dimensional Partial Differential Equations (PDEs). Here, we present a mixed Tensor Train (TT)/Quantized Tensor Train (QTT) approach for the numerical solution of time-independent Boltzm… ▽ More Tensor network techniques, known for their low-rank approximation ability that breaks the curse of dimensionality, are emerging as a foundation of new mathematical methods for ultra-fast numerical solutions of high-dimensional Partial Differential Equations (PDEs). Here, we present a mixed Tensor Train (TT)/Quantized Tensor Train (QTT) approach for the numerical solution of time-independent Boltzmann Neutron Transport equations (BNTEs) in Cartesian geometry. Discretizing a realistic three-dimensional (3D) BNTE by (i) diamond differencing, (ii) multigroup-in-energy, and (iii) discrete ordinate collocation leads to huge generalized eigenvalue problems that generally require a matrix-free approach and large computer clusters. Starting from this discretization, we construct a TT representation of the PDE fields and discrete operators, followed by a QTT representation of the TT cores and solving the tensorized generalized eigenvalue problem in a fixed-point scheme with tensor network optimization techniques. We validate our approach by applying it to two realistic examples of 3D neutron transport problems, currently solved by the PARallel TIme-dependent SN (PARTISN) solver. We demonstrate that our TT/QTT method, executed on a standard desktop computer, leads to a yottabyte compression of the memory storage, and more than 7500 times speedup with a discrepancy of less than 1e-5 when compared to the PARTISN solution. △ Less

Submitted 13 September, 2023; v1 submitted 6 September, 2023; originally announced September 2023.

Comments: 38 pages, 9 figures

MSC Class: 65

MalwareDNA: Simultaneous Classification of Malware, Malware Families, and Novel Malware

Authors: Maksim E. Eren, Manish Bhattarai, Kim Rasmussen, Boian S. Alexandrov, Charles Nicholas

Abstract: Malware is one of the most dangerous and costly cyber threats to national security and a crucial factor in modern cyber-space. However, the adoption of machine learning (ML) based solutions against malware threats has been relatively slow. Shortcomings in the existing ML approaches are likely contributing to this problem. The majority of current ML approaches ignore real-world challenges such as t… ▽ More Malware is one of the most dangerous and costly cyber threats to national security and a crucial factor in modern cyber-space. However, the adoption of machine learning (ML) based solutions against malware threats has been relatively slow. Shortcomings in the existing ML approaches are likely contributing to this problem. The majority of current ML approaches ignore real-world challenges such as the detection of novel malware. In addition, proposed ML approaches are often designed either for malware/benign-ware classification or malware family classification. Here we introduce and showcase preliminary capabilities of a new method that can perform precise identification of novel malware families, while also unifying the capability for malware/benign-ware classification and malware family classification into a single framework. △ Less

Submitted 4 September, 2023; originally announced September 2023.

Comments: Accepted at IEEE ISI 2023

Patterns and Stability of Coupled Multi-Stable Nonlinear Oscillators

Authors: Golan Bel, Boian S. Alexandrov, Alan R. Bishop, Kim Ø. Rasmussen

Abstract: Nonlinear isolated and coupled oscillators are extensively studied as prototypical nonlinear dynamics models. Much attention has been devoted to oscillator synchronization or the lack thereof. Here, we study the synchronization and stability of coupled driven-damped Helmholtz-Duffing oscillators in bi-stability regimes. We find that despite the fact that the system parameters and the driving force… ▽ More Nonlinear isolated and coupled oscillators are extensively studied as prototypical nonlinear dynamics models. Much attention has been devoted to oscillator synchronization or the lack thereof. Here, we study the synchronization and stability of coupled driven-damped Helmholtz-Duffing oscillators in bi-stability regimes. We find that despite the fact that the system parameters and the driving force are identical, the stability of the two states to spatially non-uniform perturbations is very different. Moreover, the final stable states, resulting from these spatial perturbations, are not solely dictated by the wavelength of the perturbing mode and take different spatial configurations in terms of the coupled oscillator phases. △ Less

Submitted 24 August, 2022; originally announced August 2022.

SeNMFk-SPLIT: Large Corpora Topic Modeling by Semantic Non-negative Matrix Factorization with Automatic Model Selection

Authors: Maksim E. Eren, Nick Solovyev, Manish Bhattarai, Kim Rasmussen, Charles Nicholas, Boian S. Alexandrov

Abstract: As the amount of text data continues to grow, topic modeling is serving an important role in understanding the content hidden by the overwhelming quantity of documents. One popular topic modeling approach is non-negative matrix factorization (NMF), an unsupervised machine learning (ML) method. Recently, Semantic NMF with automatic model selection (SeNMFk) has been proposed as a modification to NMF… ▽ More As the amount of text data continues to grow, topic modeling is serving an important role in understanding the content hidden by the overwhelming quantity of documents. One popular topic modeling approach is non-negative matrix factorization (NMF), an unsupervised machine learning (ML) method. Recently, Semantic NMF with automatic model selection (SeNMFk) has been proposed as a modification to NMF. In addition to heuristically estimating the number of topics, SeNMFk also incorporates the semantic structure of the text. This is performed by jointly factorizing the term frequency-inverse document frequency (TF-IDF) matrix with the co-occurrence/word-context matrix, the values of which represent the number of times two words co-occur in a predetermined window of the text. In this paper, we introduce a novel distributed method, SeNMFk-SPLIT, for semantic topic extraction suitable for large corpora. Contrary to SeNMFk, our method enables the joint factorization of large documents by decomposing the word-context and term-document matrices separately. We demonstrate the capability of SeNMFk-SPLIT by applying it to the entire artificial intelligence (AI) and ML scientific literature uploaded on arXiv. △ Less

Submitted 21 August, 2022; originally announced August 2022.

Comments: Accepted at ACM Symposium on Document Engineering 2022 (DocEng 22), 2022

Distributed Out-of-Memory SVD on CPU/GPU Architectures

Authors: Ismael Boureima, Manish Bhattarai, Maksim E. Eren, Nick Solovyev, Hristo Djidjev, Boian S. Alexandrov

Abstract: We propose an efficient, distributed, out-of-memory implementation of the truncated singular value decomposition (t-SVD) for heterogeneous (CPU+GPU) high performance computing (HPC) systems. Various implementations of SVD have been proposed, but most only estimate the singular values as an estimation of the singular vectors which can significantly increase the time and memory complexity of the alg… ▽ More We propose an efficient, distributed, out-of-memory implementation of the truncated singular value decomposition (t-SVD) for heterogeneous (CPU+GPU) high performance computing (HPC) systems. Various implementations of SVD have been proposed, but most only estimate the singular values as an estimation of the singular vectors which can significantly increase the time and memory complexity of the algorithm. In this work, we propose an implementation of SVD based on the power method, which is a truncated singular values and singular vectors estimation method. Memory utilization bottlenecks seen in the power method are typically associated with the computation of the Gram matrix $\mat{A}^T\mat{A}$, which can be significant when $\mat{A}$ is large and dense, or when $\mat{A}$ is super-large and sparse. The proposed implementation is optimized for out-of-memory problems where the memory required to factorize a given matrix is greater than the available GPU memory. We reduce the memory complexity of $\mat{A}^T\mat{A}$ by using a batching strategy where the intermediate factors are computed block by block. We also suppress I/O latency associated with both host-to-device (H2D) and device-to-host (D2H) batch copies by overlapping each batch copy with compute using CUDA streams. Furthermore, we use optimized \textit{NCCL} based communicators to reduce the latency associated with collective communications (both intra-node and inter-node). In addition, sparse and dense matrix multiplications are significantly accelerated with GPU cores (or tensors cores when available), resulting in an implementation with good scaling. We demonstrate the scalability of our distributed out of core SVD algorithm to successfully decompose dense matrix of size 1TB and sparse matrix of size 128PB with 1e-6 sparsity. △ Less

Submitted 17 August, 2022; originally announced August 2022.

Comments: Accepted at IEEE HPEC Conference 2022 with Outstanding Paper Award

FedSPLIT: One-Shot Federated Recommendation System Based on Non-negative Joint Matrix Factorization and Knowledge Distillation

Authors: Maksim E. Eren, Luke E. Richards, Manish Bhattarai, Roberto Yus, Charles Nicholas, Boian S. Alexandrov

Abstract: Non-negative matrix factorization (NMF) with missing-value completion is a well-known effective Collaborative Filtering (CF) method used to provide personalized user recommendations. However, traditional CF relies on the privacy-invasive collection of users' explicit and implicit feedback to build a central recommender model. One-shot federated learning has recently emerged as a method to mitigate… ▽ More Non-negative matrix factorization (NMF) with missing-value completion is a well-known effective Collaborative Filtering (CF) method used to provide personalized user recommendations. However, traditional CF relies on the privacy-invasive collection of users' explicit and implicit feedback to build a central recommender model. One-shot federated learning has recently emerged as a method to mitigate the privacy problem while addressing the traditional communication bottleneck of federated learning. In this paper, we present the first unsupervised one-shot federated CF implementation, named FedSPLIT, based on NMF joint factorization. In our solution, the clients first apply local CF in-parallel to build distinct client-specific recommenders. Then, the privacy-preserving local item patterns and biases from each client are shared with the processor to perform joint factorization in order to extract the global item patterns. Extracted patterns are then aggregated to each client to build the local models via knowledge distillation. In our experiments, we demonstrate the feasibility of our approach with standard recommendation datasets. FedSPLIT can obtain similar results than the state of the art (and even outperform it in certain situations) with a substantial decrease in the number of communications. △ Less

Submitted 4 May, 2022; originally announced May 2022.

Comments: Pre-print - Under review

Topic Analysis of Superconductivity Literature by Semantic Non-negative Matrix Factorization

Authors: Valentin Stanev, Erik Skau, Ichiro Takeuchi, Boian S. Alexandrov

Abstract: We utilize a recently developed topic modeling method called SeNMFk, extending the standard Non-negative Matrix Factorization (NMF) methods by incorporating the semantic structure of the text, and adding a robust system for determining the number of topics. With SeNMFk, we were able to extract coherent topics validated by human experts. From these topics, a few are relatively general and cover bro… ▽ More We utilize a recently developed topic modeling method called SeNMFk, extending the standard Non-negative Matrix Factorization (NMF) methods by incorporating the semantic structure of the text, and adding a robust system for determining the number of topics. With SeNMFk, we were able to extract coherent topics validated by human experts. From these topics, a few are relatively general and cover broad concepts, while the majority can be precisely mapped to specific scientific effects or measurement techniques. The topics also differ by ubiquity, with only three topics prevalent in almost 40 percent of the abstract, while each specific topic tends to dominate a small subset of the abstracts. These results demonstrate the ability of SeNMFk to produce a layered and nuanced analysis of large scientific corpora. △ Less

Submitted 1 December, 2021; originally announced January 2022.

Report number: LA-UR-21-3134

Quantum Annealing Algorithms for Boolean Tensor Networks

Authors: Elijah Pelofske, Georg Hahn, Daniel O'Malley, Hristo N. Djidjev, Boian S. Alexandrov

Abstract: Quantum annealers manufactured by D-Wave Systems, Inc., are computational devices capable of finding high-quality solutions of NP-hard problems. In this contribution, we explore the potential and effectiveness of such quantum annealers for computing Boolean tensor networks. Tensors offer a natural way to model high-dimensional data commonplace in many scientific fields, and representing a binary t… ▽ More Quantum annealers manufactured by D-Wave Systems, Inc., are computational devices capable of finding high-quality solutions of NP-hard problems. In this contribution, we explore the potential and effectiveness of such quantum annealers for computing Boolean tensor networks. Tensors offer a natural way to model high-dimensional data commonplace in many scientific fields, and representing a binary tensor as a Boolean tensor network is the task of expressing a tensor containing categorical (i.e., {0, 1}) values as a product of low dimensional binary tensors. A Boolean tensor network is computed by Boolean tensor decomposition, and it is usually not exact. The aim of such decomposition is to minimize the given distance measure between the high-dimensional input tensor and the product of lower-dimensional (usually three-dimensional) tensors and matrices representing the tensor network. In this paper, we introduce and analyze three general algorithms for Boolean tensor networks: Tucker, Tensor Train, and Hierarchical Tucker networks. The computation of a Boolean tensor network is reduced to a sequence of Boolean matrix factorizations, which we show can be expressed as a quadratic unconstrained binary optimization problem suitable for solving on a quantum annealer. By using a novel method we introduce called \textit{parallel quantum annealing}, we demonstrate that tensor with up to millions of elements can be decomposed efficiently using a DWave 2000Q quantum annealer. △ Less

Submitted 27 March, 2022; v1 submitted 28 July, 2021; originally announced July 2021.

Comments: Updated with new figures and fixed typos. 18 pages

Report number: LA-UR-21-27414

COVID-19 Multidimensional Kaggle Literature Organization

Authors: Maksim E. Eren, Nick Solovyev, Chris Hamer, Renee McDonald, Boian S. Alexandrov, Charles Nicholas

Abstract: The unprecedented outbreak of Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2), or COVID-19, continues to be a significant worldwide problem. As a result, a surge of new COVID-19 related research has followed suit. The growing number of publications requires document organization methods to identify relevant information. In this paper, we expand upon our previous work with clustering t… ▽ More The unprecedented outbreak of Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2), or COVID-19, continues to be a significant worldwide problem. As a result, a surge of new COVID-19 related research has followed suit. The growing number of publications requires document organization methods to identify relevant information. In this paper, we expand upon our previous work with clustering the CORD-19 dataset by applying multi-dimensional analysis methods. Tensor factorization is a powerful unsupervised learning method capable of discovering hidden patterns in a document corpus. We show that a higher-order representation of the corpus allows for the simultaneous grouping of similar articles, relevant journals, authors with similar research interests, and topic keywords. These groupings are identified within and among the latent components extracted via tensor decomposition. We further demonstrate the application of this method with a publicly available interactive visualization of the dataset. △ Less

Submitted 19 July, 2021; v1 submitted 17 July, 2021; originally announced July 2021.

Comments: Maksim E. Eren, Nick Solovyev, Chris Hamer, Renee McDonald, Boian S.Alexandrov, and Charles Nicholas. 2021. COVID-19 Multidimensional Kaggle Literature Organization. In ACM Symposium on Document Engineering 2021

Boolean Hierarchical Tucker Networks on Quantum Annealers

Authors: Elijah Pelofske, Georg Hahn, Daniel O'Malley, Hristo N. Djidjev, Boian S. Alexandrov

Abstract: Quantum annealing is an emerging technology with the potential to solve some of the computational challenges that remain unresolved as we approach an era beyond Moore's Law. In this work, we investigate the capabilities of the quantum annealers of D-Wave Systems, Inc., for computing a certain type of Boolean tensor decomposition called Boolean Hierarchical Tucker Network (BHTN). Boolean tensor dec… ▽ More Quantum annealing is an emerging technology with the potential to solve some of the computational challenges that remain unresolved as we approach an era beyond Moore's Law. In this work, we investigate the capabilities of the quantum annealers of D-Wave Systems, Inc., for computing a certain type of Boolean tensor decomposition called Boolean Hierarchical Tucker Network (BHTN). Boolean tensor decomposition problems ask for finding a decomposition of a high-dimensional tensor with categorical, [true, false], values, as a product of smaller Boolean core tensors. As the BHTN decompositions are usually not exact, we aim to approximate an input high-dimensional tensor by a product of lower-dimensional tensors such that the difference between both is minimized in some norm. We show that BHTN can be calculated as a sequence of optimization problems suitable for the D-Wave 2000Q quantum annealer. Although current technology is still fairly restricted in the problems they can address, we show that a complex problem such as BHTN can be solved efficiently and accurately. △ Less

Submitted 12 March, 2021; originally announced March 2021.

Report number: LA-UR-21-22363

Identification of Anomalous Diffusion Sources by Unsupervised Learning

Authors: Raviteja Vangara, Kim Ø. Rasmussen, Dimiter N. Petsev, Golan Bel, Boian S. Alexandrov

Abstract: Fractional Brownian motion (fBm) is a ubiquitous diffusion process in which the memory effects of the stochastic transport result in the mean squared particle displacement following a power law, $\langle {Δr}^2 \rangle \sim t^α$, where the diffusion exponent $α$ characterizes whether the transport is subdiffusive, ($α<1$), diffusive ($α= 1$), or superdiffusive, ($α>1$). Due to the abundance of fBm… ▽ More Fractional Brownian motion (fBm) is a ubiquitous diffusion process in which the memory effects of the stochastic transport result in the mean squared particle displacement following a power law, $\langle {Δr}^2 \rangle \sim t^α$, where the diffusion exponent $α$ characterizes whether the transport is subdiffusive, ($α<1$), diffusive ($α= 1$), or superdiffusive, ($α>1$). Due to the abundance of fBm processes in nature, significant efforts have been devoted to the identification and characterization of fBm sources in various phenomena. In practice, the identification of the fBm sources often relies on solving a complex and ill-posed inverse problem based on limited observed data. In the general case, the detected signals are formed by an unknown number of release sources, located at different locations and with different strengths, that act simultaneously. This means that the observed data is composed of mixtures of releases from an unknown number of sources, which makes the traditional inverse modeling approaches unreliable. Here, we report an unsupervised learning method, based on Nonnegative Matrix Factorization, that enables the identification of the unknown number of release sources as well the anomalous diffusion characteristics based on limited observed data and the general form of the corresponding fBm Green's function. We show that our method performs accurately for different types of sources and configurations with a predetermined number of sources with specific characteristics and introduced noise. △ Less

Submitted 5 October, 2020; originally announced October 2020.

Comments: published in physical review research

Journal ref: Physical Review Research, 2(2), 023248 (2020)

A Neural Network for Determination of Latent Dimensionality in Nonnegative Matrix Factorization

Authors: Benjamin T. Nebgen, Raviteja Vangara, Miguel A. Hombrados-Herrera, Svetlana Kuksova, Boian S. Alexandrov

Abstract: Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy data sets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the n… ▽ More Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy data sets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the number of hidden features, K, present in the explored data set. Unfortunately, this quantity is rarely known a priori. We utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically. NMFk performs a set of NMF simulations on an ensemble of matrices, obtained by bootstrapping the initial data set, and determines which K produces stable groups of latent features that reconstruct the initial data set well. We then train a Multi-Layer Perceptron (MLP) classifier network to determine the correct number of latent features utilizing the statistics and characteristics of the NMF solutions, obtained from NMFk. In order to train the MLP classifier, a training set of 58,660 matrices with predetermined latent features were factorized with NMFk. The MLP classifier in conjunction with NMFk maintains a greater than 95% success rate when applied to a held out test set. Additionally, when applied to two well-known benchmark data sets, the swimmer and MIT face data, NMFk/MLP correctly recovered the established number of hidden features. Finally, we compared the accuracy of our method to the ARD, AIC and Stability-based methods. △ Less

Submitted 22 June, 2020; originally announced June 2020.

Determination of Latent Dimensionality in International Trade Flow

Authors: Duc P. Truong, Erik Skau, Vladimir I. Valtchinov, Boian S. Alexandrov

Abstract: Currently, high-dimensional data is ubiquitous in data science, which necessitates the development of techniques to decompose and interpret such multidimensional (aka tensor) datasets. Finding a low dimensional representation of the data, that is, its inherent structure, is one of the approaches that can serve to understand the dynamics of low dimensional latent features hidden in the data. Nonneg… ▽ More Currently, high-dimensional data is ubiquitous in data science, which necessitates the development of techniques to decompose and interpret such multidimensional (aka tensor) datasets. Finding a low dimensional representation of the data, that is, its inherent structure, is one of the approaches that can serve to understand the dynamics of low dimensional latent features hidden in the data. Nonnegative RESCAL is one such technique, particularly well suited to analyze self-relational data, such as dynamic networks found in international trade flows. Nonnegative RESCAL computes a low dimensional tensor representation by finding the latent space containing multiple modalities. Estimating the dimensionality of this latent space is crucial for extracting meaningful latent features. Here, to determine the dimensionality of the latent space with nonnegative RESCAL, we propose a latent dimension determination method which is based on clustering of the solutions of multiple realizations of nonnegative RESCAL decompositions. We demonstrate the performance of our model selection method on synthetic data and then we apply our method to decompose a network of international trade flows data from International Monetary Fund and validate the resulting features against empirical facts from economic literature. △ Less

Submitted 28 February, 2020; originally announced March 2020.

Unsupervised Machine Learning Based on Non-Negative Tensor Factorization for Analyzing Reactive-Mixing

Authors: V. V. Vesselinov, M. K. Mudunuru, S. Karra, D. O. Malley, B. S. Alexandrov

Abstract: Analysis of reactive-diffusion simulations requires a large number of independent model runs. For each high-fidelity simulation, inputs are varied and the predicted mixing behavior is represented by changes in species concentration. It is then required to discern how the model inputs impact the mixing process. This task is challenging and typically involves interpretation of large model outputs. H… ▽ More Analysis of reactive-diffusion simulations requires a large number of independent model runs. For each high-fidelity simulation, inputs are varied and the predicted mixing behavior is represented by changes in species concentration. It is then required to discern how the model inputs impact the mixing process. This task is challenging and typically involves interpretation of large model outputs. However, the task can be automated and substantially simplified by applying Machine Learning (ML) methods. In this paper, we present an application of an unsupervised ML method (called NTFk) using Non-negative Tensor Factorization (NTF) coupled with a custom clustering procedure based on k-means to reveal hidden features in product concentration. An attractive aspect of the proposed ML method is that it ensures the extracted features are non-negative, which are important to obtain a meaningful deconstruction of the mixing processes. The ML method is applied to a large set of high-resolution FEM simulations representing reaction-diffusion processes in perturbed vortex-based velocity fields. The applied FEM ensures that species concentration are always non-negative. The simulated reaction is a fast irreversible bimolecular reaction. The reactive-diffusion model input parameters that control mixing include properties of velocity field, anisotropic dispersion, and molecular diffusion. We demonstrate the applicability of the ML method to produce a meaningful deconstruction of model outputs to discriminate between different physical processes impacting the reactants, their mixing, and the spatial distribution of the product. The presented ML analysis allowed us to identify additive features that characterize mixing behavior. △ Less

Submitted 21 February, 2019; v1 submitted 15 May, 2018; originally announced May 2018.

Comments: 34 pages

Unsupervised Phase Mapping of X-ray Diffraction Data by Nonnegative Matrix Factorization Integrated with Custom Clustering

Authors: Valentin Stanev, Velimir V. Vesselinov, A. Gilad Kusne, Graham Antoszewski, Ichiro Takeuchi, Boian S. Alexandrov

Abstract: Analyzing large X-ray diffraction (XRD) datasets is a key step in high-throughput mapping of the compositional phase diagrams of combinatorial materials libraries. Optimizing and automating this task can help accelerate the process of discovery of materials with novel and desirable properties. Here, we report a new method for pattern analysis and phase extraction of XRD datasets. The method expand… ▽ More Analyzing large X-ray diffraction (XRD) datasets is a key step in high-throughput mapping of the compositional phase diagrams of combinatorial materials libraries. Optimizing and automating this task can help accelerate the process of discovery of materials with novel and desirable properties. Here, we report a new method for pattern analysis and phase extraction of XRD datasets. The method expands the Nonnegative Matrix Factorization method, which has been used previously to analyze such datasets, by combining it with custom clustering and cross-correlation algorithms. This new method is capable of robust determination of the number of basis patterns present in the data which, in turn, enables straightforward identification of any possible peak-shifted patterns. Peak-shifting arises due to continuous change in the lattice constants as a function of composition, and is ubiquitous in XRD datasets from composition spread libraries. Successful identification of the peak-shifted patterns allows proper quantification and classification of the basis XRD patterns, which is necessary in order to decipher the contribution of each unique single-phase structure to the multi-phase regions. The process can be utilized to determine accurately the compositional phase diagram of a system under study. The presented method is applied to one synthetic and one experimental dataset, and demonstrates robust accuracy and identification abilities. △ Less

Submitted 20 February, 2018; originally announced February 2018.

Comments: 26 pages, 9 figures

Journal ref: npj Computational Materialsvolume 4, Article number: 43 (2018)

Nonnegative/binary matrix factorization with a D-Wave quantum annealer

Authors: Daniel O'Malley, Velimir V. Vesselinov, Boian S. Alexandrov, Ludmil B. Alexandrov

Abstract: D-Wave quantum annealers represent a novel computational architecture and have attracted significant interest, but have been used for few real-world computations. Machine learning has been identified as an area where quantum annealing may be useful. Here, we show that the D-Wave 2X can be effectively used as part of an unsupervised machine learning method. This method can be used to analyze large… ▽ More D-Wave quantum annealers represent a novel computational architecture and have attracted significant interest, but have been used for few real-world computations. Machine learning has been identified as an area where quantum annealing may be useful. Here, we show that the D-Wave 2X can be effectively used as part of an unsupervised machine learning method. This method can be used to analyze large datasets. The D-Wave only limits the number of features that can be extracted from the dataset. We apply this method to learn the features from a set of facial images. △ Less

Submitted 5 April, 2017; originally announced April 2017.

Nonnegative Matrix Factorization for identification of unknown number of sources emitting delayed signals

Authors: Filip L. Iliev, Valentin G. Stanev, Velimir V. Vesselinov, Boian S. Alexandrov

Abstract: Factor analysis is broadly used as a powerful unsupervised machine learning tool for reconstruction of hidden features in recorded mixtures of signals. In the case of a linear approximation, the mixtures can be decomposed by a variety of model-free Blind Source Separation (BSS) algorithms. Most of the available BSS algorithms consider an instantaneous mixing of signals, while the case when the mix… ▽ More Factor analysis is broadly used as a powerful unsupervised machine learning tool for reconstruction of hidden features in recorded mixtures of signals. In the case of a linear approximation, the mixtures can be decomposed by a variety of model-free Blind Source Separation (BSS) algorithms. Most of the available BSS algorithms consider an instantaneous mixing of signals, while the case when the mixtures are linear combinations of signals with delays is less explored. Especially difficult is the case when the number of sources of the signals with delays is unknown and has to be determined from the data as well. To address this problem, in this paper, we present a new method based on Nonnegative Matrix Factorization (NMF) that is capable of identifying: (a) the unknown number of the sources, (b) the delays and speed of propagation of the signals, and (c) the locations of the sources. Our method can be used to decompose records of mixtures of signals with delays emitted by an unknown number of sources in a nondispersive medium, based only on recorded data. This is the case, for example, when electromagnetic signals from multiple antennas are received asynchronously; or mixtures of acoustic or seismic signals recorded by sensors located at different positions; or when a shift in frequency is induced by the Doppler effect. By applying our method to synthetic datasets, we demonstrate its ability to identify the unknown number of sources as well as the waveforms, the delays, and the strengths of the signals. Using Bayesian analysis, we also evaluate estimation uncertainties and identify the region of likelihood where the positions of the sources can be found. △ Less

Submitted 23 March, 2018; v1 submitted 12 December, 2016; originally announced December 2016.

Report number: LA-UR-16-27232 MSC Class: ]68T10

Journal ref: PloS one. 2018 Mar 8;13(3):e0193974

Identification of release sources in advection-diffusion system by machine learning combined with Green function inverse method

Authors: Valentin G. Stanev, Filip L. Iliev, Scott Hansen, Velimir V. Vesselinov, Boian S. Alexandrov

Abstract: The identification of sources of advection-diffusion transport is based usually on solving complex ill-posed inverse models against the available state- variable data records. However, if there are several sources with different locations and strengths, the data records represent mixtures rather than the separate influences of the original sources. Importantly, the number of these original release… ▽ More The identification of sources of advection-diffusion transport is based usually on solving complex ill-posed inverse models against the available state- variable data records. However, if there are several sources with different locations and strengths, the data records represent mixtures rather than the separate influences of the original sources. Importantly, the number of these original release sources is typically unknown, which hinders reliability of the classical inverse-model analyses. To address this challenge, we present here a novel hybrid method for identification of the unknown number of release sources. Our hybrid method, called HNMF, couples unsupervised learning based on Nonnegative Matrix Factorization (NMF) and inverse-analysis Green functions method. HNMF synergistically performs decomposition of the recorded mixtures, finds the number of the unknown sources and uses the Green function of advection-diffusion equation to identify their characteristics. In the paper, we introduce the method and demonstrate that it is capable of identifying the advection velocity and dispersivity of the medium as well as the unknown number, locations, and properties of various sets of synthetic release sources with different space and time dependencies, based only on the recorded data. HNMF can be applied directly to any problem controlled by a partial-differential parabolic equation where mixtures of an unknown number of sources are measured at multiple locations. △ Less

Submitted 23 March, 2018; v1 submitted 12 December, 2016; originally announced December 2016.

Report number: LA-UR-16-27231 MSC Class: 68T10

Feigenbaum Cascade of Discrete Breathers in a Model of DNA

Authors: P. Maniadis, B. S. Alexandrov, A. R. Bishop, K. Ø. Rasmussen

Abstract: We demonstrate that period-doubled discrete breathers appear from the anti-continuum limit of the driven Peyrard-Bishop-Dauxois model of DNA. These novel breathers result from a stability overlap between sub-harmonic solutions of the driven Morse oscillator. Sub-harmonic breathers exist whenever a stability overlap is present within the Feigenbaum cascade to chaos and therefore an entire cascade o… ▽ More We demonstrate that period-doubled discrete breathers appear from the anti-continuum limit of the driven Peyrard-Bishop-Dauxois model of DNA. These novel breathers result from a stability overlap between sub-harmonic solutions of the driven Morse oscillator. Sub-harmonic breathers exist whenever a stability overlap is present within the Feigenbaum cascade to chaos and therefore an entire cascade of such breathers exists. This phenomenon is present in any driven lattice where the on-site potential admits sub-harmonic solutions. In DNA these breathers may have ramifications for cellular gene expression. △ Less

Submitted 11 January, 2011; v1 submitted 12 December, 2010; originally announced December 2010.

Report number: LA-UR 10-05807

Journal ref: Phys. Rev. E 83, 011904 (2011)

Robust accidental nodes and zeroes and critical quasiparticle scaling in iron-based multiband superconductors

Authors: Valentin Stanev, Boian S. Alexandrov, Predrag Nikolic, Zlatko Tesanovic

Abstract: We study multigap superconductivity, with strong angular variations of one of the gaps, as appropriate for certain iron-based high-temperature superconductors. We solve the gap equations of this model and find that the nodes or zeroes in the gap function present at Tc - although purely accidental -- typically survive down to T=0. Based on this result, we investigate the line of quantum transitions… ▽ More We study multigap superconductivity, with strong angular variations of one of the gaps, as appropriate for certain iron-based high-temperature superconductors. We solve the gap equations of this model and find that the nodes or zeroes in the gap function present at Tc - although purely accidental -- typically survive down to T=0. Based on this result, we investigate the line of quantum transitions at which gap zeroes first appear. The peculiar "zero-point" critical scaling emanating from this line dominates quasiparticle thermodynamics and transport properties over much of the phase diagram, and supplants more familiar forms of scaling associated with accidental nodes. △ Less

Submitted 19 May, 2011; v1 submitted 2 June, 2010; originally announced June 2010.

Comments: 7 pages, 8 figures, A typo corrected

Journal ref: Phys. Rev. B 84, 014505 (2011)

DNA Breathing Dynamics in the Presence of a Terahertz Field

Authors: B. S. Alexandrov, V. Gelev, A. R. Bishop, A. Usheva, K. O. Rasmussen

Abstract: We consider the influence of a terahertz field on the breathing dynamics of double-stranded DNA. We model the spontaneous formation of spatially localized openings of a damped and driven DNA chain, and find that linear instabilities lead to dynamic dimerization, while true local strand separations require a threshold amplitude mechanism. Based on our results we argue that a specific terahertz ra… ▽ More We consider the influence of a terahertz field on the breathing dynamics of double-stranded DNA. We model the spontaneous formation of spatially localized openings of a damped and driven DNA chain, and find that linear instabilities lead to dynamic dimerization, while true local strand separations require a threshold amplitude mechanism. Based on our results we argue that a specific terahertz radiation exposure may significantly affect the natural dynamics of DNA, and thereby influence intricate molecular processes involved in gene expression and DNA replication. △ Less

Submitted 28 October, 2009; originally announced October 2009.

Report number: LA-UR 09-03248

Journal ref: Physics Letters A, Volume 374, Issue 10, 2010

Branching Transport Model of Alkali-Halide Scintillators

Authors: B. S. Alexandrov, K. D. Ianakiev, P. B. Littlewood

Abstract: We measure the time dependence of the scintillator light-emission pulses in NaI(Tl) crystals at different temperatures, after activation by gamma rays. We confirm that there are two main nonexponential components to the time decay and find that their amplitude ratio shows Arrhenius temperature dependence. We explain these nonexponential components as arising from two competing mechanisms of carr… ▽ More We measure the time dependence of the scintillator light-emission pulses in NaI(Tl) crystals at different temperatures, after activation by gamma rays. We confirm that there are two main nonexponential components to the time decay and find that their amplitude ratio shows Arrhenius temperature dependence. We explain these nonexponential components as arising from two competing mechanisms of carrier transport to the Tl activation levels. The total light output of the NaI(Tl) detectors shows a linear temperature dependence explained by our model. △ Less

Submitted 25 May, 2007; originally announced May 2007.

Report number: LA-UR-07-2677

Journal ref: Nucl.Instrum.Meth.A586:432-438,2008

Classical Propagation of Light in Spatio-Temporal Periodic Media

Authors: B. S. Alexandrov, K. O. Rasmussen, A. T. Findikoglu, A. R. Bishop, I. Z. Kostadinov

Abstract: We analyze the propagation of electromagnetic waves in media where the dielectric constants undergo rapid temporal periodic modulation. Both spatially homogeneous and periodic media are studied. Fast periodic temporal modulation of the dielectric constant of a homogeneous medium leads to existence of photonic band-gap like phenomena. In the presence of both spatial and tem- poral periodicity the… ▽ More We analyze the propagation of electromagnetic waves in media where the dielectric constants undergo rapid temporal periodic modulation. Both spatially homogeneous and periodic media are studied. Fast periodic temporal modulation of the dielectric constant of a homogeneous medium leads to existence of photonic band-gap like phenomena. In the presence of both spatial and tem- poral periodicity the electromagnetic spectrum is described in a four-dimensional cube, defining an effective Brillouin zone. In the case of incommensurability between space and time periodicities, completely dispersed point spectra exist. △ Less

Submitted 18 October, 2006; originally announced October 2006.

Temperature behavior of NaI (Tl) scintillation detectors

Authors: K. D. Ianakiev, B. S. Alexandrov, P. B. Littlewood, M. C. Browne

Abstract: It is a familiar fact that the total measured light yield of NaI (Tl) detectors is a nonlinear function of temperature. Here we present new experimental data for the temperature behavior of doped NaI(Tl) scintillators that instead shows a linear dependence of light output over a wide temperature range- including that for outdoor applications. The shape of the light pulse shows in general two dec… ▽ More It is a familiar fact that the total measured light yield of NaI (Tl) detectors is a nonlinear function of temperature. Here we present new experimental data for the temperature behavior of doped NaI(Tl) scintillators that instead shows a linear dependence of light output over a wide temperature range- including that for outdoor applications. The shape of the light pulse shows in general two decay processes: a single dominant process above room temperature and two decay time constants below. We show that redistribution of the intensities is temperature-dependent; the second (slow) decay component is negligible at room temperatures, but, by -20C, it contributes up to 40 percents of the total light and has a duration of several microseconds. We discuss the profound effect this new understanding of the light output has on the pulse height analysis instrumentation. We introduce a theoretical model to explain the experimental results. In addition, we describe a unique technique for correcting both amplitude and shape temperature changes inside the NaI(Tl) detector package. △ Less

Submitted 30 May, 2006; originally announced May 2006.

Report number: LA-UR-06-3417

Journal ref: Nucl.Instrum.Meth.A607:432-438,2009

Bubble Statistics and Dynamics in Double-Stranded DNA

Authors: B. S. Alexandrov, L. T. Wille, K. O. Rasmussen, A. R. Bishop, K. B. Blagoev

Abstract: The dynamical properties of double-stranded DNA are studied in the framework of the Peyrard-Bishop-Dauxois model using Langevin dynamics. Our simulations are analyzed in terms of two probability functions describing coherently localized separations ("bubbles") of the double strand. We find that the resulting bubble distributions are more sharply peaked at the active sites than found in thermodyn… ▽ More The dynamical properties of double-stranded DNA are studied in the framework of the Peyrard-Bishop-Dauxois model using Langevin dynamics. Our simulations are analyzed in terms of two probability functions describing coherently localized separations ("bubbles") of the double strand. We find that the resulting bubble distributions are more sharply peaked at the active sites than found in thermodynamically obtained distributions. Our analysis ascribes this to the fact that the bubble life-times significantly afects the distribution function. We find that certain base-pair sequences promote long-lived bubbles and we argue that this is due to a length scale competition between the nonlinearity and disorder present in the system. △ Less

Submitted 24 January, 2006; originally announced January 2006.

Journal ref: Phys. Rev. E 74, 050901(R) (2006)