-
Towards Edge-Based Data Lake Architecture for Intelligent Transportation System
Authors:
Danilo Fernandes,
Douglas L. L. Moura,
Gean Santos,
Geymerson S. Ramos,
Fabiane Queiroz,
Andre L. L. Aquino
Abstract:
The rapid urbanization growth has underscored the need for innovative solutions to enhance transportation efficiency and safety. Intelligent Transportation Systems (ITS) have emerged as a promising solution in this context. However, analyzing and processing the massive and intricate data generated by ITS presents significant challenges for traditional data processing systems. This work proposes an…
▽ More
The rapid urbanization growth has underscored the need for innovative solutions to enhance transportation efficiency and safety. Intelligent Transportation Systems (ITS) have emerged as a promising solution in this context. However, analyzing and processing the massive and intricate data generated by ITS presents significant challenges for traditional data processing systems. This work proposes an Edge-based Data Lake Architecture to integrate and analyze the complex data from ITS efficiently. The architecture offers scalability, fault tolerance, and performance, improving decision-making and enhancing innovative services for a more intelligent transportation ecosystem. We demonstrate the effectiveness of the architecture through an analysis of three different use cases: (i) Vehicular Sensor Network, (ii) Mobile Network, and (iii) Driver Identification applications.
△ Less
Submitted 4 September, 2024;
originally announced September 2024.
-
A flexible algorithm to offload DAG applications for edge computing
Authors:
Gabriel F. C. de Queiroz,
José F. de Rezende,
Valmir C. Barbosa
Abstract:
Multi-access Edge Computing (MEC) is an enabling technology to leverage new network applications, such as virtual/augmented reality, by providing faster task processing at the network edge. This is done by deploying servers closer to the end users to run the network applications. These applications are often intensive in terms of task processing, memory usage, and communication; thus mobile device…
▽ More
Multi-access Edge Computing (MEC) is an enabling technology to leverage new network applications, such as virtual/augmented reality, by providing faster task processing at the network edge. This is done by deploying servers closer to the end users to run the network applications. These applications are often intensive in terms of task processing, memory usage, and communication; thus mobile devices may take a long time or even not be able to run them efficiently. By transferring (offloading) the execution of these applications to the servers at the network edge, it is possible to achieve a lower completion time (makespan) and meet application requirements. However, offloading multiple entire applications to the edge server can overwhelm its hardware and communication channel, as well as underutilize the mobile devices' hardware. In this paper, network applications are modeled as Directed Acyclic Graphs (DAGs) and partitioned into tasks, and only part of these tasks are offloaded to the edge server. This is the DAG application partitioning and offloading problem, which is known to be NP-hard. To approximate its solution, this paper proposes the FlexDO algorithm. FlexDO combines a greedy phase with a permutation phase to find a set of offloading decisions, and then chooses the one that achieves the shortest makespan. FlexDO is compared with a proposal from the literature and two baseline decisions, considering realistic DAG applications extracted from the Alibaba Cluster Trace Program. Results show that FlexDO is consistently only 3.9% to 8.9% above the optimal makespan in all test scenarios, which include different levels of CPU availability, a multi-user case, and different communication channel transmission rates. FlexDO outperforms both baseline solutions by a wide margin, and is three times closer to the optimal makespan than its competitor.
△ Less
Submitted 15 June, 2023;
originally announced June 2023.
-
A matheuristic approach for the $b$-coloring problem using integer programming and a multi-start multi-greedy randomized metaheuristic
Authors:
Rafael A. Melo,
Michell F. Queiroz,
Marcio C. Santos
Abstract:
Given a graph $G=(V,E)$, the $b$-coloring problem consists in attributing a color to every vertex in $V$ such that adjacent vertices receive different colors, every color has a $b$-vertex, and the number of colors is maximized. A $b$-vertex is a vertex adjacent to vertices colored with all used colors but its own. The $b$-coloring problem is known to be NP-Hard and its optimal solution determines…
▽ More
Given a graph $G=(V,E)$, the $b$-coloring problem consists in attributing a color to every vertex in $V$ such that adjacent vertices receive different colors, every color has a $b$-vertex, and the number of colors is maximized. A $b$-vertex is a vertex adjacent to vertices colored with all used colors but its own. The $b$-coloring problem is known to be NP-Hard and its optimal solution determines the $b$-chromatic number of $G$, denoted $χ_b(G)$. This paper presents an integer programming formulation and a very effective multi-greedy randomized heuristic which can be used in a multi-start metaheuristic. In addition, a matheuristic approach is proposed combining the multi-start multi-greedy randomized metaheuristic with a MIP (mixed integer programming) based local search procedure using the integer programming formulation. Computational experiments establish the proposed multi-start metaheuristic as very effective in generating high quality solutions, along with the matheuristic approach successfully improving several of those results. Moreover, the computational results show that the multi-start metaheuristic outperforms a state-of-the-art hybrid evolutionary metaheuristic for a subset of the large instances which were previously considered in the literature. An additional contribution of this work is the proposal of a benchmark instance set, which consists of newly generated instances as well as others available in the literature for classical graph problems, with the aim of standardizing computational comparisons of approaches for the $b$-coloring problem in future works.
△ Less
Submitted 18 February, 2021;
originally announced February 2021.
-
A New Similarity Space Tailored for Supervised Deep Metric Learning
Authors:
Pedro H. Barros,
Fabiane Queiroz,
Flavio Figueredo,
Jefersson A. dos Santos,
Heitor S. Ramos
Abstract:
We propose a novel deep metric learning method. Differently from many works on this area, we defined a novel latent space obtained through an autoencoder. The new space, namely S-space, is divided into different regions that describe the positions where pairs of objects are similar/dissimilar. We locate makers to identify these regions. We estimate the similarities between objects through a kernel…
▽ More
We propose a novel deep metric learning method. Differently from many works on this area, we defined a novel latent space obtained through an autoencoder. The new space, namely S-space, is divided into different regions that describe the positions where pairs of objects are similar/dissimilar. We locate makers to identify these regions. We estimate the similarities between objects through a kernel-based t-student distribution to measure the markers' distance and the new data representation. In our approach, we simultaneously estimate the markers' position in the S-space and represent the objects in the same space. Moreover, we propose a new regularization function to avoid similar markers to collapse altogether. We present evidences that our proposal can represent complex spaces, for instance, when groups of similar objects are located in disjoint regions. We compare our proposal to 9 different distance metric learning approaches (four of them are based on deep-learning) on 28 real-world heterogeneous datasets. According to the four quantitative metrics used, our method overcomes all the nine strategies from the literature.
△ Less
Submitted 18 November, 2020; v1 submitted 16 November, 2020;
originally announced November 2020.
-
Good Usability Practices in Scientific Software Development
Authors:
Francisco Queiroz,
Raniere Silva,
Jonah Miller,
Sandor Brockhauser,
Hans Fangohr
Abstract:
Scientific software often presents very particular requirements regarding usability, which is often completely overlooked in this setting. As computational science has emerged as its own discipline, distinct from theoretical and experimental science, it has put new requirements on future scientific software developments. In this paper, we discuss the background of these problems and introduce nine…
▽ More
Scientific software often presents very particular requirements regarding usability, which is often completely overlooked in this setting. As computational science has emerged as its own discipline, distinct from theoretical and experimental science, it has put new requirements on future scientific software developments. In this paper, we discuss the background of these problems and introduce nine aspects of good usability. We also highlight best practices for each aspect with an emphasis on applications in computational science.
△ Less
Submitted 31 August, 2017;
originally announced September 2017.
