-
Assessing Code Understanding in LLMs
Authors:
Cosimo Laneve,
Alvise Spanò,
Dalila Ressi,
Sabina Rossi,
Michele Bugliesi
Abstract:
We present an empirical evaluation of Large Language Models in code understanding associated with non-trivial, semantic-preserving program transformations such as copy propagation or constant folding. Our findings show that LLMs fail to judge semantic equivalence in approximately 41\% of cases when no context is provided and in 29\% when given a simple generic context. To improve accuracy, we advo…
▽ More
We present an empirical evaluation of Large Language Models in code understanding associated with non-trivial, semantic-preserving program transformations such as copy propagation or constant folding. Our findings show that LLMs fail to judge semantic equivalence in approximately 41\% of cases when no context is provided and in 29\% when given a simple generic context. To improve accuracy, we advocate integrating LLMs with code-optimization tools to enhance training and facilitate more robust program understanding.
△ Less
Submitted 31 March, 2025;
originally announced April 2025.
-
Vulnerability Detection in Ethereum Smart Contracts via Machine Learning: A Qualitative Analysis
Authors:
Dalila Ressi,
Alvise Spanò,
Lorenzo Benetollo,
Carla Piazza,
Michele Bugliesi,
Sabina Rossi
Abstract:
Smart contracts are central to a myriad of critical blockchain applications, from financial transactions to supply chain management. However, their adoption is hindered by security vulnerabilities that can result in significant financial losses. Most vulnerability detection tools and methods available nowadays leverage either static analysis methods or machine learning. Unfortunately, as valuable…
▽ More
Smart contracts are central to a myriad of critical blockchain applications, from financial transactions to supply chain management. However, their adoption is hindered by security vulnerabilities that can result in significant financial losses. Most vulnerability detection tools and methods available nowadays leverage either static analysis methods or machine learning. Unfortunately, as valuable as they are, both approaches suffer from limitations that make them only partially effective. In this survey, we analyze the state of the art in machine-learning vulnerability detection for Ethereum smart contracts, by categorizing existing tools and methodologies, evaluating them, and highlighting their limitations. Our critical assessment unveils issues such as restricted vulnerability coverage and dataset construction flaws, providing us with new metrics to overcome the difficulties that restrain a sound comparison of existing solutions. Driven by our findings, we discuss best practices to enhance the accuracy, scope, and efficiency of vulnerability detection in smart contracts. Our guidelines address the known flaws while at the same time opening new avenues for research and development. By shedding light on current challenges and offering novel directions for improvement, we contribute to the advancement of secure smart contract development and blockchain technology as a whole.
△ Less
Submitted 26 July, 2024;
originally announced July 2024.
-
Smart Contract Languages: a comparative analysis
Authors:
Massimo Bartoletti,
Lorenzo Benetollo,
Michele Bugliesi,
Silvia Crafa,
Giacomo Dal Sasso,
Roberto Pettinau,
Andrea Pinna,
Mattia Piras,
Sabina Rossi,
Stefano Salis,
Alvise Spanò,
Viacheslav Tkachenko,
Roberto Tonelli,
Roberto Zunino
Abstract:
Smart contracts have played a pivotal role in the evolution of blockchains and Decentralized Applications (DApps). As DApps continue to gain widespread adoption, multiple smart contract languages have been and are being made available to developers, each with its distinctive features, strengths, and weaknesses. In this paper, we examine the smart contract languages used in major blockchain platfor…
▽ More
Smart contracts have played a pivotal role in the evolution of blockchains and Decentralized Applications (DApps). As DApps continue to gain widespread adoption, multiple smart contract languages have been and are being made available to developers, each with its distinctive features, strengths, and weaknesses. In this paper, we examine the smart contract languages used in major blockchain platforms, with the goal of providing a comprehensive assessment of their main properties. Our analysis targets the programming languages rather than the underlying architecture: as a result, while we do consider the interplay between language design and blockchain model, our main focus remains on language-specific features such as usability, programming style, safety and security. To conduct our assessment, we propose an original benchmark which encompasses a wide, yet manageable, spectrum of key use cases that cut across all the smart contract languages under examination.
△ Less
Submitted 8 August, 2024; v1 submitted 5 April, 2024;
originally announced April 2024.
