-
Supporting architecture evaluation for ATAM scenarios with LLMs
Authors:
Rafael Capilla,
J. Andrés Díaz-Pace,
Yamid Ramírez,
Jennifer Pérez,
Vanessa Rodríguez-Horcajo
Abstract:
Architecture evaluation methods have long been used to evaluate software designs. Several evaluation methods have been proposed and used to analyze tradeoffs between different quality attributes. Having competing qualities leads to conflicts for selecting which quality-attribute scenarios are the most suitable ones that an architecture should tackle and for prioritizing the scenarios required by t…
▽ More
Architecture evaluation methods have long been used to evaluate software designs. Several evaluation methods have been proposed and used to analyze tradeoffs between different quality attributes. Having competing qualities leads to conflicts for selecting which quality-attribute scenarios are the most suitable ones that an architecture should tackle and for prioritizing the scenarios required by the stakeholders. In this context, architecture evaluation is carried out manually, often involving long brainstorming sessions to decide which are the most adequate quality scenarios. To reduce this effort and make the assessment and selection of scenarios more efficient, we suggest the usage of LLMs to partially automate evaluation activities. As a first step to validate this hypothesis, this work studies MS Copilot as an LLM tool to analyze quality scenarios suggested by students in a software architecture course and compares the students' results with the assessment provided by the LLM. Our initial study reveals that the LLM produces in most cases better and more accurate results regarding the risks, sensitivity points and tradeoff analysis of the quality scenarios. Overall, the use of generative AI has the potential to partially automate and support the architecture evaluation tasks, improving the human decision-making process.
△ Less
Submitted 30 May, 2025;
originally announced June 2025.
-
Towards Assessing Spread in Sets of Software Architecture Designs
Authors:
Vittorio Cortellessa,
J. Andres Diaz-Pace,
Daniele Di Pompeo,
Michele Tucci
Abstract:
Several approaches have recently used automated techniques to generate architecture design alternatives by means of optimization techniques. These approaches aim at improving an initial architecture with respect to quality aspects, such as performance, reliability, or maintainability. In this context, each optimization experiment usually produces a different set of architecture alternatives that i…
▽ More
Several approaches have recently used automated techniques to generate architecture design alternatives by means of optimization techniques. These approaches aim at improving an initial architecture with respect to quality aspects, such as performance, reliability, or maintainability. In this context, each optimization experiment usually produces a different set of architecture alternatives that is characterized by specific settings. As a consequence, the designer is left with the task of comparing such sets to identify the settings that lead to better solution sets for the problem. To assess the quality of solution sets, multi-objective optimization commonly relies on quality indicators. Among these, the quality indicator for the maximum spread estimates the diversity of the generated alternatives, providing a measure of how much of the solution space has been explored. However, the maximum spread indicator is computed only on the objective space and does not consider architectural information (e.g., components structure, design decisions) from the architectural space. In this paper, we propose a quality indicator for the spread that assesses the diversity of alternatives by taking into account architectural features. To compute the spread, we rely on a notion of distance between alternatives according to the way they were generated during the optimization. We demonstrate how our architectural quality indicator can be applied to a dataset from the literature.
△ Less
Submitted 29 February, 2024;
originally announced February 2024.
-
Hyperparameter optimization of hp-greedy reduced basis for gravitational wave surrogates
Authors:
Franco Cerino,
Andrés Diaz-Pace,
Emmanuel Tassone,
Manuel Tiglio,
Atuel Villegas
Abstract:
In a previous work we introduced, in the context of gravitational wave science, an initial study on an automated domain-decomposition approach for reduced basis through hp-greedy refinement. The approach constructs local reduced bases of lower dimensionality than global ones, with the same or higher accuracy. These ``light'' local bases should imply both faster evaluations when predicting new wave…
▽ More
In a previous work we introduced, in the context of gravitational wave science, an initial study on an automated domain-decomposition approach for reduced basis through hp-greedy refinement. The approach constructs local reduced bases of lower dimensionality than global ones, with the same or higher accuracy. These ``light'' local bases should imply both faster evaluations when predicting new waveforms and faster data analysis, in particular faster statistical inference (the forward and inverse problems, respectively). In this approach, however, we have previously found important dependence on several hyperparameters, which do not appear in global reduced basis. This naturally leads to the problem of hyperparameter optimization (HPO), which is the subject of this paper. We tackle the problem through a Bayesian optimization, and show its superiority when compared to grid or random searches. We find that for gravitational waves from the collision of two spinning but non-precessing black holes, for the same accuracy, local hp-greedy reduced bases with HPO have a lower dimensionality of up to $4 \times$ for the cases here studied, depending on the desired accuracy. This factor should directly translate in a parameter estimation speedup, for instance. Such acceleration might help in the near real-time requirements for electromagnetic counterparts of gravitational waves from compact binary coalescences. In addition, we find that the Bayesian approach used in this paper for HPO is two orders of magnitude faster than, for example, a grid search, with about a $100 \times$ acceleration. The code developed for this project is available as open source from public repositories.
△ Less
Submitted 23 October, 2023;
originally announced October 2023.
-
On the Role of Search Budgets in Model-Based Software Refactoring Optimization
Authors:
J. Andres Diaz-Pace,
Daniele Di Pompeo,
Michele Tucci
Abstract:
Software model optimization is a process that automatically generates design alternatives aimed at improving quantifiable non-functional properties of software systems, such as performance and reliability. Multi-objective evolutionary algorithms effectively help designers identify trade-offs among the desired non-functional properties. To reduce the use of computational resources, this work examin…
▽ More
Software model optimization is a process that automatically generates design alternatives aimed at improving quantifiable non-functional properties of software systems, such as performance and reliability. Multi-objective evolutionary algorithms effectively help designers identify trade-offs among the desired non-functional properties. To reduce the use of computational resources, this work examines the impact of implementing a search budget to limit the search for design alternatives. In particular, we analyze how time budgets affect the quality of Pareto fronts by utilizing quality indicators and exploring the structural features of the generated design alternatives. This study identifies distinct behavioral differences among evolutionary algorithms when a search budget is implemented. It further reveals that design alternatives generated under a budget are structurally different from those produced without one. Additionally, we offer recommendations for designers on selecting algorithms in relation to time constraints, thereby facilitating the effective application of automated refactoring to improve non-functional properties.
△ Less
Submitted 8 May, 2025; v1 submitted 29 August, 2023;
originally announced August 2023.
-
Introducing Interactions in Multi-Objective Optimization of Software Architectures
Authors:
Vittorio Cortellessa,
J. Andres Diaz-Pace,
Daniele Di Pompeo,
Sebastian Frank,
Pooyan Jamshidi,
Michele Tucci,
André van Hoorn
Abstract:
Software architecture optimization aims to enhance non-functional attributes like performance and reliability while meeting functional requirements. Multi-objective optimization employs metaheuristic search techniques, such as genetic algorithms, to explore feasible architectural changes and propose alternatives to designers. However, this resource-intensive process may not always align with pract…
▽ More
Software architecture optimization aims to enhance non-functional attributes like performance and reliability while meeting functional requirements. Multi-objective optimization employs metaheuristic search techniques, such as genetic algorithms, to explore feasible architectural changes and propose alternatives to designers. However, this resource-intensive process may not always align with practical constraints. This study investigates the impact of designer interactions on multi-objective software architecture optimization. Designers can intervene at intermediate points in the fully automated optimization process, making choices that guide exploration towards more desirable solutions. Through several controlled experiments as well as an initial user study (14 subjects), we compare this interactive approach with a fully automated optimization process, which serves as a baseline. The findings demonstrate that designer interactions lead to a more focused solution space, resulting in improved architectural quality. By directing the search towards regions of interest, the interaction uncovers architectures that remain unexplored in the fully automated process. In the user study, participants found that our interactive approach provides a better trade-off between sufficient exploration of the solution space and the required computation time.
△ Less
Submitted 19 December, 2024; v1 submitted 29 August, 2023;
originally announced August 2023.
-
An automated parameter domain decomposition approach for gravitational wave surrogates using hp-greedy refinement
Authors:
Franco Cerino,
J. Andrés Diaz-Pace,
Manuel Tiglio
Abstract:
We introduce hp-greedy, a refinement approach for building gravitational wave surrogates as an extension of the standard reduced basis framework. Our proposal is data-driven, with a domain decomposition of the parameter space, local reduced basis, and a binary tree as the resulting structure, which are obtained in an automated way. When compared to the standard global reduced basis approach, the n…
▽ More
We introduce hp-greedy, a refinement approach for building gravitational wave surrogates as an extension of the standard reduced basis framework. Our proposal is data-driven, with a domain decomposition of the parameter space, local reduced basis, and a binary tree as the resulting structure, which are obtained in an automated way. When compared to the standard global reduced basis approach, the numerical simulations of our proposal show three salient features: i) representations of lower dimension with no loss of accuracy, ii) a significantly higher accuracy for a fixed maximum dimensionality of the basis, in some cases by orders of magnitude, and iii) results that depend on the reduced basis seed choice used by the refinement algorithm. We first illustrate the key parts of our approach with a toy model and then present a more realistic use case of gravitational waves emitted by the collision of two spinning, non-precessing black holes. We discuss performance aspects of hp-greedy, such as overfitting with respect to the depth of the tree structure, and other hyperparameter dependences. As two direct applications of the proposed hp-greedy refinement, we envision: i) a further acceleration of statistical inference, which might be complementary to focused reduced-order quadratures, and ii) the search of gravitational waves through clustering and nearest neighbors.
△ Less
Submitted 16 December, 2022;
originally announced December 2022.
-
Capturing social media expressions during the COVID-19 pandemic in Argentina and forecasting mental health and emotions
Authors:
Antonela Tommasel,
Andres Diaz-Pace,
Juan Manuel Rodriguez,
Daniela Godoy
Abstract:
Purpose. We present an approach for forecasting mental health conditions and emotions of a given population during the COVID-19 pandemic in Argentina based on language expressions used in social media. This approach permits anticipating high prevalence periods in short- to medium-term time horizons. Design. Mental health conditions and emotions are captured via markers, which link social media con…
▽ More
Purpose. We present an approach for forecasting mental health conditions and emotions of a given population during the COVID-19 pandemic in Argentina based on language expressions used in social media. This approach permits anticipating high prevalence periods in short- to medium-term time horizons. Design. Mental health conditions and emotions are captured via markers, which link social media contents with lexicons. First, we build descriptive timelines for decision makers to monitor the evolution of markers, and their correlation with crisis events. Second, we model the timelines as time series, and support their forecasting, which in turn serve to identify high prevalence points for the estimated markers. Findings. Results showed that different time series forecasting strategies offer different capabilities. In the best scenario, the emergence of high prevalence periods of emotions and mental health disorders can be satisfactorily predicted with a neural network strategy, even when limited data is available in early stages of a crisis (e.g., 7 days). Originality. Although there have been efforts in the literature to predict mental states of individuals, the analysis of mental health at the collective level has received scarce attention. We take a step forward by proposing a forecasting approach for analyzing the mental health of a given population (or group of individuals) at a larger scale. Practical implications. We believe that this work contributes to a better understanding of how psychological processes related to crisis manifest in social media, being a valuable asset for the design, implementation and monitoring of health prevention and communication policies.
△ Less
Submitted 19 February, 2021; v1 submitted 12 January, 2021;
originally announced January 2021.
-
Towards a Service-oriented Platform for Intelligent Apps in Intermediate Cities
Authors:
J. Andres Diaz-Pace,
Luis Berdun,
Alejandro Zunino,
Silvia Schiaffino
Abstract:
Smart cities are a growing trend in many cities in Argentina. In particular, the so-called intermediate cities present a context and requirements different from those of large cities with respect to smart cities. One aspect of relevance is to encourage the development of applications (generally for mobile devices) that enable citizens to take advantage of data and services normally associated with…
▽ More
Smart cities are a growing trend in many cities in Argentina. In particular, the so-called intermediate cities present a context and requirements different from those of large cities with respect to smart cities. One aspect of relevance is to encourage the development of applications (generally for mobile devices) that enable citizens to take advantage of data and services normally associated with the city, for example, in the urban mobility domain. In this work, a platform is proposed for intermediate cities that provide "high level" services and that allow the construction of software applications that consume those services. Our platform-centric strategy focused aims to integrate systems and heterogeneous data sources, and provide "intelligent" services to different applications. Examples of these services include: construction of user profiles, recommending local events, and collaborative sensing based on data mining techniques, among others. In this work, the design of this platform (currently in progress) is described, and experiences of applications for urban mobility are discussed, which are being migrated in the form of reusable services provided by the platform
△ Less
Submitted 27 August, 2018;
originally announced August 2018.
-
Towards Anticipation of Architectural Smells using Link Prediction Techniques
Authors:
J. Andrés Díaz-Pace,
Antonela Tommasel,
Daniela Godoy
Abstract:
Software systems naturally evolve, and this evolution often brings design problems that cause system degradation. Architectural smells are typical symptoms of such problems, and several of these smells are related to undesired dependencies among modules. The early detection of these smells is important for developers, because they can plan ahead for maintenance or refactoring efforts, thus prevent…
▽ More
Software systems naturally evolve, and this evolution often brings design problems that cause system degradation. Architectural smells are typical symptoms of such problems, and several of these smells are related to undesired dependencies among modules. The early detection of these smells is important for developers, because they can plan ahead for maintenance or refactoring efforts, thus preventing system degradation. Existing tools for identifying architectural smells can detect the smells once they exist in the source code. This means that their undesired dependencies are already created. In this work, we explore a forward-looking approach that is able to infer groups of likely module dependencies that can anticipate architectural smells in a future system version. Our approach considers the current module structure as a network, along with information from previous versions, and applies link prediction techniques (from the field of social network analysis). In particular, we focus on dependency-related smells, such as Cyclic Dependency and Hublike Dependency, which fit well with the link prediction model. An initial evaluation with two open-source projects shows that, under certain considerations, the predictions of our approach are satisfactory. Furthermore, the approach can be extended to other types of dependency-based smells or metrics.
△ Less
Submitted 20 August, 2018;
originally announced August 2018.
-
Can Network Analysis Techniques help to Predict Design Dependencies? An Initial Study
Authors:
J. Andrés Díaz-Pace,
Antonela Tommasel,
Daniela Godoy
Abstract:
The degree of dependencies among the modules of a software system is a key attribute to characterize its design structure and its ability to evolve over time. Several design problems are often correlated with undesired dependencies among modules. Being able to anticipate those problems is important for developers, so they can plan early for maintenance and refactoring efforts. However, existing to…
▽ More
The degree of dependencies among the modules of a software system is a key attribute to characterize its design structure and its ability to evolve over time. Several design problems are often correlated with undesired dependencies among modules. Being able to anticipate those problems is important for developers, so they can plan early for maintenance and refactoring efforts. However, existing tools are limited to detecting undesired dependencies once they appeared in the system. In this work, we investigate whether module dependencies can be predicted (before they actually appear). Since the module structure can be regarded as a network, i.e, a dependency graph, we leverage on network features to analyze the dynamics of such a structure. In particular, we apply link prediction techniques for this task. We conducted an evaluation on two Java projects across several versions, using link prediction and machine learning techniques, and assessed their performance for identifying new dependencies from a project version to the next one. The results, although preliminary, show that the link prediction approach is feasible for package dependencies. Also, this work opens opportunities for further development of software-specific strategies for dependency prediction.
△ Less
Submitted 8 August, 2018;
originally announced August 2018.