Title:Diagnosing Violations of State-based Specifications in iCFTL

Abstract:As modern software systems grow in complexity and operate in dynamic environments, the need for runtime analysis techniques becomes a more critical part of the verification and validation process. Runtime verification monitors the runtime system behaviour by checking whether an execution trace - a sequence of recorded events - satisfies a given specification, yielding a Boolean or quantitative verdict. However, when a specification is violated, such a verdict is often insufficient to understand why the violation happened. To fill this gap, diagnostics approaches aim to produce more informative verdicts. In this paper, we address the problem of generating informative verdicts for violated Inter-procedural Control-Flow Temporal Logic (iCFTL) specifications that express constraints over program variable values. We propose a diagnostic approach based on backward data-flow analysis to statically determine the relevant statements contributing to the specification violation. Using this analysis, we instrument the program to produce enriched execution traces. Using the enriched execution traces, we perform the runtime analysis and identify the statements whose execution led to the specification violation. We implemented our approach in a prototype tool, iCFTL-Diagnostics, and evaluated it on 112 specifications across 10 software projects. Our tool achieves 90% precision in identifying relevant statements for 100 of the 112 specifications. It reduces the number of lines that have to be inspected for diagnosing a violation by at least 90%. In terms of computational cost, iCFTL-Diagnostics generates a diagnosis within 7 min, and requires no more than 25 MB of memory. The instrumentation required to support diagnostics incurs an execution time overhead of less than 30% and a memory overhead below 20%.