-
DILLEMA: Diffusion and Large Language Models for Multi-Modal Augmentation
Authors:
Luciano Baresi,
Davide Yi Xian Hu,
Muhammad Irfan Mas'udi,
Giovanni Quattrocchi
Abstract:
Ensuring the robustness of deep learning models requires comprehensive and diverse testing. Existing approaches, often based on simple data augmentation techniques or generative adversarial networks, are limited in producing realistic and varied test cases. To address these limitations, we present a novel framework for testing vision neural networks that leverages Large Language Models and control…
▽ More
Ensuring the robustness of deep learning models requires comprehensive and diverse testing. Existing approaches, often based on simple data augmentation techniques or generative adversarial networks, are limited in producing realistic and varied test cases. To address these limitations, we present a novel framework for testing vision neural networks that leverages Large Language Models and control-conditioned Diffusion Models to generate synthetic, high-fidelity test cases. Our approach begins by translating images into detailed textual descriptions using a captioning model, allowing the language model to identify modifiable aspects of the image and generate counterfactual descriptions. These descriptions are then used to produce new test images through a text-to-image diffusion process that preserves spatial consistency and maintains the critical elements of the scene. We demonstrate the effectiveness of our method using two datasets: ImageNet1K for image classification and SHIFT for semantic segmentation in autonomous driving. The results show that our approach can generate significant test cases that reveal weaknesses and improve the robustness of the model through targeted retraining. We conducted a human assessment using Mechanical Turk to validate the generated images. The responses from the participants confirmed, with high agreement among the voters, that our approach produces valid and realistic images.
△ Less
Submitted 5 February, 2025;
originally announced February 2025.
-
Efficient Domain Augmentation for Autonomous Driving Testing Using Diffusion Models
Authors:
Luciano Baresi,
Davide Yi Xian Hu,
Andrea Stocco,
Paolo Tonella
Abstract:
Simulation-based testing is widely used to assess the reliability of Autonomous Driving Systems (ADS), but its effectiveness is limited by the operational design domain (ODD) conditions available in such simulators. To address this limitation, in this work, we explore the integration of generative artificial intelligence techniques with physics-based simulators to enhance ADS system-level testing.…
▽ More
Simulation-based testing is widely used to assess the reliability of Autonomous Driving Systems (ADS), but its effectiveness is limited by the operational design domain (ODD) conditions available in such simulators. To address this limitation, in this work, we explore the integration of generative artificial intelligence techniques with physics-based simulators to enhance ADS system-level testing. Our study evaluates the effectiveness and computational overhead of three generative strategies based on diffusion models, namely instruction-editing, inpainting, and inpainting with refinement. Specifically, we assess these techniques' capabilities to produce augmented simulator-generated images of driving scenarios representing new ODDs. We employ a novel automated detector for invalid inputs based on semantic segmentation to ensure semantic preservation and realism of the neural generated images. We then perform system-level testing to evaluate the ADS's generalization ability to newly synthesized ODDs. Our findings show that diffusion models help increase the ODD coverage for system-level testing of ADS. Our automated semantic validator achieved a percentage of false positives as low as 3%, retaining the correctness and quality of the generated images for testing. Our approach successfully identified new ADS system failures before real-world testing.
△ Less
Submitted 17 February, 2025; v1 submitted 20 September, 2024;
originally announced September 2024.
-
Dependency-aware Resource Allocation for Serverless Functions at the Edge
Authors:
Luciano Baresi,
Giovanni Quattrocchi,
Inacio Ticongolo
Abstract:
Serverless computing allows developers to break their code into small components, known as functions, which are automatically managed by a service provider. Being lightweight and modular, serverless functions have been increasingly employed in edge computing, where quick responses and adaptability are key to meeting strict latency requirements. In particular, edge nodes are intrinsically resource-…
▽ More
Serverless computing allows developers to break their code into small components, known as functions, which are automatically managed by a service provider. Being lightweight and modular, serverless functions have been increasingly employed in edge computing, where quick responses and adaptability are key to meeting strict latency requirements. In particular, edge nodes are intrinsically resource-constrained, and efficient resource allocation strategies are crucial for optimizing their usage. Different approaches exist in the literature, but they often overlook the dependencies among functions, that is, how and when functions invoke other functions, obtaining suboptimal results.
This paper presents NEPTUNE+, a dependency-aware resource (CPU cores) allocation solution for serverless functions deployed at the edge. The approach extends NEPTUNE, an existing framework for managing edge infrastructures, with a new theoretical model and control algorithm that take dependencies into account function. We evaluated NEPTUNE+ by using three applications and it is able to allocate up to 42% fewer cores compared to NEPTUNE.
△ Less
Submitted 22 September, 2023;
originally announced January 2024.
-
A Multi-faceted Analysis of the Performance Variability of Virtual Machines
Authors:
Luciano Baresi,
Tommaso Dolci,
Giovanni Quattrocchi,
Nicholas Rasi
Abstract:
Cloud computing and virtualization solutions allow one to rent the virtual machines (VMs) needed to run applications on a pay-per-use basis, but rented VMs do not offer any guarantee on their performance. Cloud platforms are known to be affected by performance variability, but a better understanding is still required. This paper moves in that direction and presents an in-depth, multi-faceted study…
▽ More
Cloud computing and virtualization solutions allow one to rent the virtual machines (VMs) needed to run applications on a pay-per-use basis, but rented VMs do not offer any guarantee on their performance. Cloud platforms are known to be affected by performance variability, but a better understanding is still required. This paper moves in that direction and presents an in-depth, multi-faceted study on the performance variability of VMs. Unlike previous studies, our assessment covers a wide range of factors: 16 VM types from 4 well-known cloud providers, 10 benchmarks, and 28 different metrics. We present four new contributions. First, we introduce a new benchmark suite (VMBS) that let researchers and practitioners systematically collect a diverse set of performance data. Second, we present a new indicator, called Variability Indicator, that allows for measuring variability in the performance of VMs. Third, we illustrate an analysis of the collected data across four different dimensions: resources, isolation, time, and cost. Fourth, we present multiple predictive models based on Machine Learning that aim to forecast future performance and detect time patterns. Our experiments provide important insights on the resource variability of VMs, highlighting differences and similarities between various cloud providers. To the best of our knowledge, this is the widest analysis ever conducted on the topic.
△ Less
Submitted 21 September, 2023;
originally announced September 2023.
-
A Qualitative and Quantitative Analysis of Container Engines
Authors:
Luciano Baresi,
Giovanni Quattrocchi,
Nicholas Rasi
Abstract:
Containerization is a virtualization technique that allows one to create and run executables consistently on any infrastructure. Compared to virtual machines, containers are lighter since they do not bundle a (guest) operating system but they share its kernel, and they only include the files, libraries, and dependencies that are required to properly execute a process. In the past few years, multip…
▽ More
Containerization is a virtualization technique that allows one to create and run executables consistently on any infrastructure. Compared to virtual machines, containers are lighter since they do not bundle a (guest) operating system but they share its kernel, and they only include the files, libraries, and dependencies that are required to properly execute a process. In the past few years, multiple container engines (i.e., tools for configuring, executing, and managing containers) have been developed ranging from some that are ``general purpose'', and mostly employed for Cloud executions, to others that are built for specific contexts, namely Internet of Things and High-Performance Computing. Given the importance of this technology for many practitioners and researchers, this paper analyses six state-of-the-art container engines and compares them through a comprehensive study of their characteristics and performance. The results are organized around 10 findings that aim to help the readers understand the differences among the technologies and help them choose the best approach for their needs.
△ Less
Submitted 7 March, 2023;
originally announced March 2023.
-
Microservice Architecture Practices and Experience: a Focused Look on Docker Configuration Files
Authors:
Luciano Baresi,
Giovanni Quattrocchi,
Damian Andrew Tamburri
Abstract:
Cloud applications are more and more microservice-oriented, but a concrete charting of the microservices architecture landscape -- namely, the space of technical options available for microservice software architects in their decision-making -- is still very much lacking, thereby limiting the ability of software architects to properly evaluate their architectural decisions with sound experiential…
▽ More
Cloud applications are more and more microservice-oriented, but a concrete charting of the microservices architecture landscape -- namely, the space of technical options available for microservice software architects in their decision-making -- is still very much lacking, thereby limiting the ability of software architects to properly evaluate their architectural decisions with sound experiential devices and/or practical design principles. On the one hand, Microservices are fine-grained, loosely coupled services that communicate through lightweight protocols. On the other hand, each microservice can use a different software stack, be deployed and scaled independently or even executed in different containers, which provide isolation and a wide-range of configuration options but also offer unforeseeable architectural interactions and under-explored architecture smells, with such experience captured mainly in software repositories where such solutions are cycled.
This paper adopts a mining software repositories (MSR) approach to capture the practice within the microservice architecture landscape, by eliciting and analysing Docker configuration files, being Docker the leading technical device to design for, and implement modern microservices. Our analysis of Docker-based microservices gives an interesting summary of the current state of microservices practice and experience. Conversely, observing that all our datapoints have their own shape and characteristics, we conclude that further comparative assessment with industrial systems is needed to better address the recurring positive principles and patterns around microservices.
△ Less
Submitted 6 December, 2022;
originally announced December 2022.
-
Training and Serving Machine Learning Models at Scale
Authors:
Luciano Baresi,
Giovanni Quattrocchi
Abstract:
In recent years, Web services are becoming more and more intelligent (e.g., in understanding user preferences) thanks to the integration of components that rely on Machine Learning (ML). Before users can interact (inference phase) with an ML-based service (ML-Service), the underlying ML model must learn (training phase) from existing data, a process that requires long-lasting batch computations. T…
▽ More
In recent years, Web services are becoming more and more intelligent (e.g., in understanding user preferences) thanks to the integration of components that rely on Machine Learning (ML). Before users can interact (inference phase) with an ML-based service (ML-Service), the underlying ML model must learn (training phase) from existing data, a process that requires long-lasting batch computations. The management of these two, diverse phases is complex and meeting time and quality requirements can hardly be done with manual approaches.
This paper highlights some of the major issues in managing ML-services in both training and inference modes and presents some initial solutions that are able to meet set requirements with minimum user inputs. A preliminary evaluation demonstrates that our solutions allow these systems to become more efficient and predictable with respect to their response time and accuracy.
△ Less
Submitted 10 November, 2022;
originally announced November 2022.
-
A Declarative Modelling Framework for the Deployment and Management of Blockchain Applications
Authors:
Luciano Baresi,
Giovanni Quattrocchi,
Damian Andrew Tamburri,
Luca Terracciano
Abstract:
The deployment and management of Blockchain applications require non-trivial efforts given the unique characteristics of their infrastructure (i.e., immutability) and the complexity of the software systems being executed. The operation of Blockchain applications is still based on ad-hoc solutions that are error-prone, difficult to maintain and evolve, and do not manage their interactions with othe…
▽ More
The deployment and management of Blockchain applications require non-trivial efforts given the unique characteristics of their infrastructure (i.e., immutability) and the complexity of the software systems being executed. The operation of Blockchain applications is still based on ad-hoc solutions that are error-prone, difficult to maintain and evolve, and do not manage their interactions with other infrastructures (e.g., a Cloud backend).
This paper proposes KATENA, a framework for the deployment and management of Blockchain applications. In particular, it focuses on applications that are compatible with Ethereum, a popular general-purpose Blockchain technology. KATENA provides i) a metamodel for defining Blockchain applications, ii) a set of processes to automate the deployment and management of defined models, and iii) an implementation of the approach based on TOSCA, a standard language for Infrastructure-as-Code, and xOpera, a TOSCA-compatible orchestrator. To evaluate the approach, we applied KATENA to model and deploy three real-world Blockchain applications, and showed that our solution reduces the amount of code required for their operations up to $82.7\%$.
△ Less
Submitted 12 September, 2022;
originally announced September 2022.
-
NEPTUNE: Network- and GPU-aware Management of Serverless Functions at the Edge
Authors:
Luciano Baresi,
Davide Yi Xian Hu,
Giovanni Quattrocchi,
Luca Terracciano
Abstract:
Nowadays a wide range of applications is constrained by low-latency requirements that cloud infrastructures cannot meet. Multi-access Edge Computing (MEC) has been proposed as the reference architecture for executing applications closer to users and reduce latency, but new challenges arise: edge nodes are resource-constrained, the workload can vary significantly since users are nomadic, and task c…
▽ More
Nowadays a wide range of applications is constrained by low-latency requirements that cloud infrastructures cannot meet. Multi-access Edge Computing (MEC) has been proposed as the reference architecture for executing applications closer to users and reduce latency, but new challenges arise: edge nodes are resource-constrained, the workload can vary significantly since users are nomadic, and task complexity is increasing (e.g., machine learning inference). To overcome these problems, the paper presents NEPTUNE, a serverless-based framework for managing complex MEC solutions. NEPTUNE i) places functions on edge nodes according to user locations, ii) avoids the saturation of single nodes, iii) exploits GPUs when available, and iv) allocates resources (CPU cores) dynamically to meet foreseen execution times. A prototype, built on top of K3S, was used to evaluate NEPTUNE on a set of experiments that demonstrate a significant reduction in terms of response time, network overhead, and resource consumption compared to three state-of-the-art approaches.
△ Less
Submitted 9 May, 2022;
originally announced May 2022.
-
Cloud Deployment Tradeoffs for the Analysis of Spatially-Distributed Systems of Internet-of-Things
Authors:
Christos Tsigkanos,
Martin Garriga,
Luciano Baresi,
Carlo Ghezzi
Abstract:
Internet-enabled things and devices operating in the physical world are increasingly integrated in modern distributed systems, supporting functionalities that require assurances that certain critical requirements are satisfied by the overall system. We focus here on spatially-distributed Internet-of-Things systems such as smart environments, where the dynamics of spatial distribution of entities i…
▽ More
Internet-enabled things and devices operating in the physical world are increasingly integrated in modern distributed systems, supporting functionalities that require assurances that certain critical requirements are satisfied by the overall system. We focus here on spatially-distributed Internet-of-Things systems such as smart environments, where the dynamics of spatial distribution of entities in the system is crucial to requirements satisfaction. Analysis techniques need to be in place while systems operate to ensure that requirements are fulfilled. This may be achieved by keeping a model of the system at runtime, monitoring events that lead to changes in the spatial environment, and performing analysis. This computationally-intensive runtime assurance method cannot be supported by resource-constrained devices that populate the space and must be offloaded to the cloud. However, challenges arise regarding resource allocation and cost, especially when the workload is unknown at the system's design time. As such, it may be difficult or even impossible to guarantee application service level agreements, e.g., on response times. To this end, we instantiate spatial verification processes, integrating them to the service layer of an IoT-cloud architecture based on microservices. We propose several cloud deployments for such an architecture for assurance of spatial requirements -- based on virtual machines, containers, and the recent Functions-as-a-Service paradigm. Then, we assess deployments' tradeoffs in terms of elasticity, performance and cost by using a workload scenario from a known dataset of taxis roaming in Beijing. We argue that the approach can be replicated in the design process of similar kinds of spatially distributed Internet-of-Things systems.
△ Less
Submitted 23 April, 2020;
originally announced April 2020.
-
Feature-Model-Guided Online Learning for Self-Adaptive Systems
Authors:
Andreas Metzger,
Clément Quinton,
Zoltán Ádám Mann,
Luciano Baresi,
Klaus Pohl
Abstract:
A self-adaptive system can modify its own structure and behavior at runtime based on its perception of the environment, of itself and of its requirements. To develop a self-adaptive system, software developers codify knowledge about the system and its environment, as well as how adaptation actions impact on the system. However, the codified knowledge may be insufficient due to design time uncertai…
▽ More
A self-adaptive system can modify its own structure and behavior at runtime based on its perception of the environment, of itself and of its requirements. To develop a self-adaptive system, software developers codify knowledge about the system and its environment, as well as how adaptation actions impact on the system. However, the codified knowledge may be insufficient due to design time uncertainty, and thus a self-adaptive system may execute adaptation actions that do not have the desired effect. Online learning is an emerging approach to address design time uncertainty by employing machine learning at runtime. Online learning accumulates knowledge at runtime by, for instance, exploring not-yet executed adaptation actions. We address two specific problems with respect to online learning for self-adaptive systems. First, the number of possible adaptation actions can be very large. Existing online learning techniques randomly explore the possible adaptation actions, but this can lead to slow convergence of the learning process. Second, the possible adaptation actions can change as a result of system evolution. Existing online learning techniques are unaware of these changes and thus do not explore new adaptation actions, but explore adaptation actions that are no longer valid. We propose using feature models to give structure to the set of adaptation actions and thereby guide the exploration process during online learning. Experimental results involving four real-world systems suggest that considering the hierarchical structure of feature models may speed up convergence by 7.2% on average. Considering the differences between feature models before and after an evolution step may speed up convergence by 64.6% on average. [...]
△ Less
Submitted 22 July, 2019;
originally announced July 2019.
