-
GraphAide: Advanced Graph-Assisted Query and Reasoning System
Authors:
Sumit Purohit,
George Chin,
Patrick S Mackey,
Joseph A Cottam
Abstract:
Curating knowledge from multiple siloed sources that contain both structured and unstructured data is a major challenge in many real-world applications. Pattern matching and querying represent fundamental tasks in modern data analytics that leverage this curated knowledge. The development of such applications necessitates overcoming several research challenges, including data extraction, named ent…
▽ More
Curating knowledge from multiple siloed sources that contain both structured and unstructured data is a major challenge in many real-world applications. Pattern matching and querying represent fundamental tasks in modern data analytics that leverage this curated knowledge. The development of such applications necessitates overcoming several research challenges, including data extraction, named entity recognition, data modeling, and designing query interfaces. Moreover, the explainability of these functionalities is critical for their broader adoption.
The emergence of Large Language Models (LLMs) has accelerated the development lifecycle of new capabilities. Nonetheless, there is an ongoing need for domain-specific tools tailored to user activities. The creation of digital assistants has gained considerable traction in recent years, with LLMs offering a promising avenue to develop such assistants utilizing domain-specific knowledge and assumptions.
In this context, we introduce an advanced query and reasoning system, GraphAide, which constructs a knowledge graph (KG) from diverse sources and allows to query and reason over the resulting KG. GraphAide harnesses both the KG and LLMs to rapidly develop domain-specific digital assistants. It integrates design patterns from retrieval augmented generation (RAG) and the semantic web to create an agentic LLM application. GraphAide underscores the potential for streamlined and efficient development of specialized digital assistants, thereby enhancing their applicability across various domains.
△ Less
Submitted 29 October, 2024;
originally announced November 2024.
-
Semantic Property Graph for Scalable Knowledge Graph Analytics
Authors:
Sumit Purohit,
Nhuy Van,
George Chin
Abstract:
Graphs are a natural and fundamental representation of describing the activities, relationships, and evolution of various complex systems. Many domains such as communication, citation, procurement, biology, social media, and transportation can be modeled as a set of entities and their relationships. Resource Description Framework (RDF) and Labeled Property Graph (LPG) are two of the most used data…
▽ More
Graphs are a natural and fundamental representation of describing the activities, relationships, and evolution of various complex systems. Many domains such as communication, citation, procurement, biology, social media, and transportation can be modeled as a set of entities and their relationships. Resource Description Framework (RDF) and Labeled Property Graph (LPG) are two of the most used data models to encode information in a graph. Both models are similar in terms of using basic graph elements such as nodes and edges but differ in terms of modeling approach, expressibility, serialization, and target applications. RDF is a flexible data exchange model for expressing information about entities but it tends to a have high memory footprint and inefficient storage, which does not make it a natural choice to perform scalable graph analytics. In contrast, LPG has gained traction as a reliable model in performing scalable graph analytic tasks such as sub-graph matching, network alignment, and real-time knowledge graph query. It provides efficient storage, fast traversal, and flexibility to model various real-world domains. At the same time, the LPGs lack the support of a formal knowledge representation such as an ontology to provide automated knowledge inference. We propose Semantic Property Graph (SPG) as a logical projection of reified RDF into LPG model. SPG continues to use RDF ontology to define type hierarchy of the projected graph and validate it against a given ontology. We present a framework to convert reified RDF graphs into SPG using two different computing environments. We also present cloud-based graph migration capabilities using Amazon Web Services.
△ Less
Submitted 24 September, 2020; v1 submitted 15 September, 2020;
originally announced September 2020.
-
ITeM: Independent Temporal Motifs to Summarize and Compare Temporal Networks
Authors:
Sumit Purohit,
Lawrence B. Holder,
George Chin
Abstract:
Networks are a fundamental and flexible way of representing various complex systems. Many domains such as communication, citation, procurement, biology, social media, and transportation can be modeled as a set of entities and their relationships. Temporal networks are a specialization of general networks where the temporal evolution of the system is as important to understand as the structure of t…
▽ More
Networks are a fundamental and flexible way of representing various complex systems. Many domains such as communication, citation, procurement, biology, social media, and transportation can be modeled as a set of entities and their relationships. Temporal networks are a specialization of general networks where the temporal evolution of the system is as important to understand as the structure of the entities and relationships. We present the Independent Temporal Motif (ITeM) to characterize temporal graphs from different domains. The ITeMs are edge-disjoint temporal motifs that can be used to model the structure and the evolution of the graph. For a given temporal graph, we produce a feature vector of ITeM frequencies and apply this distribution to the task of measuring the similarity of temporal graphs. We show that ITeM has higher accuracy than other motif frequency-based approaches. We define various metrics based on ITeM that reveal salient properties of a temporal network. We also present importance sampling as a method for efficiently estimating the ITeM counts. We evaluate our approach on both synthetic and real temporal networks.
△ Less
Submitted 5 August, 2020; v1 submitted 19 February, 2020;
originally announced February 2020.
-
A Chronological Edge-Driven Approach to Temporal Subgraph Isomorphism
Authors:
Patrick Mackey,
Katherine Porterfield,
Erin Fitzhenry,
Sutanay Choudhury,
George Chin Jr
Abstract:
Many real world networks are considered temporal networks, in which the chronological ordering of the edges has importance to the meaning of the data. Performing temporal subgraph matching on such graphs requires the edges in the subgraphs to match the order of the temporal graph motif we are searching for. Previous methods for solving this rely on the use of static subgraph matching to find poten…
▽ More
Many real world networks are considered temporal networks, in which the chronological ordering of the edges has importance to the meaning of the data. Performing temporal subgraph matching on such graphs requires the edges in the subgraphs to match the order of the temporal graph motif we are searching for. Previous methods for solving this rely on the use of static subgraph matching to find potential matches first, before filtering them based on edge order to find the true temporal matches. We present a new algorithm for temporal subgraph isomorphism that performs the subgraph matching directly on the chronologically sorted edges. By restricting our search to only the subgraphs with chronologically correct edges, we can improve the performance of the algorithm significantly. We present experimental timing results to show significant performance improvements on publicly available datasets for a number of different temporal query graph motifs with four or more nodes. We also demonstrate a practical example of how temporal subgraph isomorphism can produce more meaningful results than traditional static subgraph searches.
△ Less
Submitted 24 January, 2018;
originally announced January 2018.
-
A Selectivity based approach to Continuous Pattern Detection in Streaming Graphs
Authors:
Sutanay Choudhury,
Lawrence Holder,
George Chin,
Khushbu Agarwal,
John Feo
Abstract:
Cyber security is one of the most significant technical challenges in current times. Detecting adversarial activities, prevention of theft of intellectual properties and customer data is a high priority for corporations and government agencies around the world. Cyber defenders need to analyze massive-scale, high-resolution network flows to identify, categorize, and mitigate attacks involving netwo…
▽ More
Cyber security is one of the most significant technical challenges in current times. Detecting adversarial activities, prevention of theft of intellectual properties and customer data is a high priority for corporations and government agencies around the world. Cyber defenders need to analyze massive-scale, high-resolution network flows to identify, categorize, and mitigate attacks involving networks spanning institutional and national boundaries. Many of the cyber attacks can be described as subgraph patterns, with prominent examples being insider infiltrations (path queries), denial of service (parallel paths) and malicious spreads (tree queries). This motivates us to explore subgraph matching on streaming graphs in a continuous setting. The novelty of our work lies in using the subgraph distributional statistics collected from the streaming graph to determine the query processing strategy. We introduce a "Lazy Search" algorithm where the search strategy is decided on a vertex-to-vertex basis depending on the likelihood of a match in the vertex neighborhood. We also propose a metric named "Relative Selectivity" that is used to select between different query processing strategies. Our experiments performed on real online news, network traffic stream and a synthetic social network benchmark demonstrate 10-100x speedups over selectivity agnostic approaches.
△ Less
Submitted 3 March, 2015;
originally announced March 2015.
-
Query Optimization for Dynamic Graphs
Authors:
Sutanay Choudhury,
Lawrence Holder,
George Chin,
Patrick Mackey,
Khushbu Agarwal,
John Feo
Abstract:
Given a query graph that represents a pattern of interest, the emerging pattern detection problem can be viewed as a continuous query problem on a dynamic graph. We present an incremental algorithm for continuous query processing on dynamic graphs. The algorithm is based on the concept of query decomposition; we decompose a query graph into smaller subgraphs and assemble the result of sub-queries…
▽ More
Given a query graph that represents a pattern of interest, the emerging pattern detection problem can be viewed as a continuous query problem on a dynamic graph. We present an incremental algorithm for continuous query processing on dynamic graphs. The algorithm is based on the concept of query decomposition; we decompose a query graph into smaller subgraphs and assemble the result of sub-queries to find complete matches with the specified query. The novelty of our work lies in using the subgraph distributional statistics collected from the dynamic graph to generate the decomposition. We introduce a "Lazy Search" algorithm where the search strategy is decided on a vertex-to-vertex basis depending on the likelihood of a match in the vertex neighborhood. We also propose a metric named "Relative Selectivity" that is used to select between different query decomposition strategies. Our experiments performed on real online news, network traffic stream and a synthetic social network benchmark demonstrate 10-100x speedups over competing approaches.
△ Less
Submitted 14 July, 2014;
originally announced July 2014.
-
StreamWorks - A system for Dynamic Graph Search
Authors:
Sutanay Choudhury,
Lawrence Holder,
George Chin,
Abhik Ray,
Sherman Beus,
John Feo
Abstract:
Acting on time-critical events by processing ever growing social media, news or cyber data streams is a major technical challenge. Many of these data sources can be modeled as multi-relational graphs. Mining and searching for subgraph patterns in a continuous setting requires an efficient approach to incremental graph search. The goal of our work is to enable real-time search capabilities for grap…
▽ More
Acting on time-critical events by processing ever growing social media, news or cyber data streams is a major technical challenge. Many of these data sources can be modeled as multi-relational graphs. Mining and searching for subgraph patterns in a continuous setting requires an efficient approach to incremental graph search. The goal of our work is to enable real-time search capabilities for graph databases. This demonstration will present a dynamic graph query system that leverages the structural and semantic characteristics of the underlying multi-relational graph.
△ Less
Submitted 11 June, 2013;
originally announced June 2013.
-
Fast Search for Dynamic Multi-Relational Graphs
Authors:
Sutanay Choudhury,
Lawrence Holder,
George Chin,
John Feo
Abstract:
Acting on time-critical events by processing ever growing social media or news streams is a major technical challenge. Many of these data sources can be modeled as multi-relational graphs. Continuous queries or techniques to search for rare events that typically arise in monitoring applications have been studied extensively for relational databases. This work is dedicated to answer the question th…
▽ More
Acting on time-critical events by processing ever growing social media or news streams is a major technical challenge. Many of these data sources can be modeled as multi-relational graphs. Continuous queries or techniques to search for rare events that typically arise in monitoring applications have been studied extensively for relational databases. This work is dedicated to answer the question that emerges naturally: how can we efficiently execute a continuous query on a dynamic graph? This paper presents an exact subgraph search algorithm that exploits the temporal characteristics of representative queries for online news or social media monitoring. The algorithm is based on a novel data structure called the Subgraph Join Tree (SJ-Tree) that leverages the structural and semantic characteristics of the underlying multi-relational graph. The paper concludes with extensive experimentation on several real-world datasets that demonstrates the validity of this approach.
△ Less
Submitted 11 June, 2013;
originally announced June 2013.
-
Scalable Triadic Analysis of Large-Scale Graphs: Multi-Core vs. Multi- Processor vs. Multi-Threaded Shared Memory Architectures
Authors:
George Chin Jr.,
Andres Marquez,
Sutanay Choudhury,
John Feo
Abstract:
Triadic analysis encompasses a useful set of graph mining methods that are centered on the concept of a triad, which is a subgraph of three nodes. Such methods are often applied in the social sciences as well as many other diverse fields. Triadic methods commonly operate on a triad census that counts the number of triads of every possible edge configuration in a graph. Like other graph algorithms,…
▽ More
Triadic analysis encompasses a useful set of graph mining methods that are centered on the concept of a triad, which is a subgraph of three nodes. Such methods are often applied in the social sciences as well as many other diverse fields. Triadic methods commonly operate on a triad census that counts the number of triads of every possible edge configuration in a graph. Like other graph algorithms, triadic census algorithms do not scale well when graphs reach tens of millions to billions of nodes. To enable the triadic analysis of large-scale graphs, we developed and optimized a triad census algorithm to efficiently execute on shared memory architectures. We then conducted performance evaluations of the parallel triad census algorithm on three specific systems: Cray XMT, HP Superdome, and AMD multi-core NUMA machine. These three systems have shared memory architectures but with markedly different hardware capabilities to manage parallelism.
△ Less
Submitted 27 September, 2012;
originally announced September 2012.
-
Continuous Queries for Multi-Relational Graphs
Authors:
Sutanay Choudhury,
Lawrence B. Holder,
Abhik Ray,
George Chin Jr.,
John T. Feo
Abstract:
Acting on time-critical events by processing ever growing social media or news streams is a major technical challenge. Many of these data sources can be modeled as multi-relational graphs. Continuous queries or techniques to search for rare events that typically arise in monitoring applications have been studied extensively for relational databases. This work is dedicated to answer the question th…
▽ More
Acting on time-critical events by processing ever growing social media or news streams is a major technical challenge. Many of these data sources can be modeled as multi-relational graphs. Continuous queries or techniques to search for rare events that typically arise in monitoring applications have been studied extensively for relational databases. This work is dedicated to answer the question that emerges naturally: how can we efficiently execute a continuous query on a dynamic graph? This paper presents an exact subgraph search algorithm that exploits the temporal characteristics of representative queries for online news or social media monitoring. The algorithm is based on a novel data structure called the Subgraph Join Tree (SJ-Tree) that leverages the structural and semantic characteristics of the underlying multi-relational graph. The paper concludes with extensive experimentation on several real-world datasets that demonstrates the validity of this approach.
△ Less
Submitted 8 March, 2013; v1 submitted 10 September, 2012;
originally announced September 2012.
-
Large-scale continuous subgraph queries on streams
Authors:
Sutanay Choudhury,
Lawrence Holder,
George Chin,
John Feo
Abstract:
Graph pattern matching involves finding exact or approximate matches for a query subgraph in a larger graph. It has been studied extensively and has strong applications in domains such as computer vision, computational biology, social networks, security and finance. The problem of exact graph pattern matching is often described in terms of subgraph isomorphism which is NP-complete. The exponential…
▽ More
Graph pattern matching involves finding exact or approximate matches for a query subgraph in a larger graph. It has been studied extensively and has strong applications in domains such as computer vision, computational biology, social networks, security and finance. The problem of exact graph pattern matching is often described in terms of subgraph isomorphism which is NP-complete. The exponential growth in streaming data from online social networks, news and video streams and the continual need for situational awareness motivates a solution for finding patterns in streaming updates. This is also the prime driver for the real-time analytics market. Development of incremental algorithms for graph pattern matching on streaming inputs to a continually evolving graph is a nascent area of research. Some of the challenges associated with this problem are the same as found in continuous query (CQ) evaluation on streaming databases. This paper reviews some of the representative work from the exhaustively researched field of CQ systems and identifies important semantics, constraints and architectural features that are also appropriate for HPC systems performing real-time graph analytics. For each of these features we present a brief discussion of the challenge encountered in the database realm, the approach to the solution and state their relevance in a high-performance, streaming graph processing framework.
△ Less
Submitted 31 July, 2012;
originally announced August 2012.
