-
Conformal Prediction in Multi-User Settings: An Evaluation
Authors:
Enrique Garcia-Ceja,
Luciano Garcia-Banuelos,
Nicolas Jourdan
Abstract:
Typically, machine learning models are trained and evaluated without making any distinction between users (e.g, using traditional hold-out and cross-validation). However, this produces inaccurate performance metrics estimates in multi-user settings. That is, situations where the data were collected by multiple users with different characteristics (e.g., age, gender, height, etc.) which is very com…
▽ More
Typically, machine learning models are trained and evaluated without making any distinction between users (e.g, using traditional hold-out and cross-validation). However, this produces inaccurate performance metrics estimates in multi-user settings. That is, situations where the data were collected by multiple users with different characteristics (e.g., age, gender, height, etc.) which is very common in user computer interaction and medical applications. For these types of scenarios model evaluation strategies that provide better performance estimates have been proposed such as mixed, user-independent, user-dependent, and user-adaptive models. Although those strategies are better suited for multi-user systems, they are typically assessed with respect to performance metrics that capture the overall behavior of the models and do not provide any performance guarantees for individual predictions nor they provide any feedback about the predictions' uncertainty. In order to overcome those limitations, in this work we evaluated the conformal prediction framework in several multi-user settings. Conformal prediction is a model agnostic method that provides confidence guarantees on the predictions, thus, increasing the trustworthiness and robustness of the models. We conducted extensive experiments using different evaluation strategies and found significant differences in terms of conformal performance measures. We also proposed several visualizations based on matrices, graphs, and charts that capture different aspects of the resulting prediction sets.
△ Less
Submitted 8 December, 2023;
originally announced December 2023.
-
Bootstrapping Generalization of Process Models Discovered From Event Data
Authors:
Artem Polyvyanyy,
Alistair Moffat,
Luciano García-Bañuelos
Abstract:
Process mining extracts value from the traces recorded in the event logs of IT-systems, with process discovery the task of inferring a process model for a log emitted by some unknown system. Generalization is one of the quality criteria applied to process models to quantify how well the model describes future executions of the system. Generalization is also perhaps the least understood of those cr…
▽ More
Process mining extracts value from the traces recorded in the event logs of IT-systems, with process discovery the task of inferring a process model for a log emitted by some unknown system. Generalization is one of the quality criteria applied to process models to quantify how well the model describes future executions of the system. Generalization is also perhaps the least understood of those criteria, with that lack primarily a consequence of it measuring properties over the entire future behavior of the system when the only available sample of behavior is that provided by the log. In this paper, we apply a bootstrap approach from computational statistics, allowing us to define an estimator of the model's generalization based on the log it was discovered from. We show that standard process mining assumptions lead to a consistent estimator that makes fewer errors as the quality of the log increases. Experiments confirm the ability of the approach to support industry-scale data-driven systems engineering.
△ Less
Submitted 25 March, 2022; v1 submitted 8 July, 2021;
originally announced July 2021.
-
Entropia: A Family of Entropy-Based Conformance Checking Measures for Process Mining
Authors:
Artem Polyvyanyy,
Hanan Alkhammash,
Claudio Di Ciccio,
Luciano García-Bañuelos,
Anna Kalenkova,
Sander J. J. Leemans,
Jan Mendling,
Alistair Moffat,
Matthias Weidlich
Abstract:
This paper presents a command-line tool, called Entropia, that implements a family of conformance checking measures for process mining founded on the notion of entropy from information theory. The measures allow quantifying classical non-deterministic and stochastic precision and recall quality criteria for process models automatically discovered from traces executed by IT-systems and recorded in…
▽ More
This paper presents a command-line tool, called Entropia, that implements a family of conformance checking measures for process mining founded on the notion of entropy from information theory. The measures allow quantifying classical non-deterministic and stochastic precision and recall quality criteria for process models automatically discovered from traces executed by IT-systems and recorded in their event logs. A process model has "good" precision with respect to the log it was discovered from if it does not encode many traces that are not part of the log, and has "good" recall if it encodes most of the traces from the log. By definition, the measures possess useful properties and can often be computed quickly.
△ Less
Submitted 29 September, 2020; v1 submitted 21 August, 2020;
originally announced August 2020.
-
An Entropic Relevance Measure for Stochastic Conformance Checking in Process Mining
Authors:
Artem Polyvyanyy,
Alistair Moffat,
Luciano García-Bañuelos
Abstract:
Given an event log as a collection of recorded real-world process traces, process mining aims to automatically construct a process model that is both simple and provides a useful explanation of the traces. Conformance checking techniques are then employed to characterize and quantify commonalities and discrepancies between the log's traces and the candidate models. Recent approaches to conformance…
▽ More
Given an event log as a collection of recorded real-world process traces, process mining aims to automatically construct a process model that is both simple and provides a useful explanation of the traces. Conformance checking techniques are then employed to characterize and quantify commonalities and discrepancies between the log's traces and the candidate models. Recent approaches to conformance checking acknowledge that the elements being compared are inherently stochastic - for example, some traces occur frequently and others infrequently - and seek to incorporate this knowledge in their analyses.
Here we present an entropic relevance measure for stochastic conformance checking, computed as the average number of bits required to compress each of the log's traces, based on the structure and information about relative likelihoods provided by the model. The measure penalizes traces from the event log not captured by the model and traces described by the model but absent in the event log, thus addressing both precision and recall quality criteria at the same time. We further show that entropic relevance is computable in time linear in the size of the log, and provide evaluation outcomes that demonstrate the feasibility of using the new approach in industrial settings.
△ Less
Submitted 26 August, 2020; v1 submitted 17 July, 2020;
originally announced July 2020.
-
Interpreted Execution of Business Process Models on Blockchain
Authors:
Orlenys López-Pintado,
Marlon Dumas,
Luciano García-Bañuelos,
Ingo Weber
Abstract:
Blockchain technology provides a tamper-proof mechanism to execute inter-organizational business processes involving mutually untrusted parties. Existing approaches to blockchain-based process execution are based on code generation. In these approaches, a process model is compiled into one or more smart contracts, which are then deployed on a blockchain platform. Given the immutability of the depl…
▽ More
Blockchain technology provides a tamper-proof mechanism to execute inter-organizational business processes involving mutually untrusted parties. Existing approaches to blockchain-based process execution are based on code generation. In these approaches, a process model is compiled into one or more smart contracts, which are then deployed on a blockchain platform. Given the immutability of the deployed smart contracts, these compiled approaches ensure that all process instances conform to the process model. However, this advantage comes at the price of inflexibility. Any changes to the process model require the redeployment of the smart contracts (a costly operation). In addition, changes cannot be applied to running process instances. To address this lack of flexibility, this paper presents an interpreter of BPMN process models based on dynamic data structures. The proposed interpreter is embedded in a business process execution system with a modular multi-layered architecture, supporting the creation, execution, monitoring and dynamic update of process instances. For efficiency purposes, the interpreter relies on compact bitmap-based encodings of process models. An experimental evaluation shows that the proposed interpreted approach achieves comparable or lower costs relative to existing compiled approaches.
△ Less
Submitted 4 June, 2019;
originally announced June 2019.
-
Business Process Privacy Analysis in Pleak
Authors:
Aivo Toots,
Reedik Tuuling,
Maksym Yerokhin,
Marlon Dumas,
Luciano García-Bañuelos,
Peeter Laud,
Raimundas Matulevičius,
Alisa Pankova,
Martin Pettai,
Pille Pullonen,
Jake Tom
Abstract:
Pleak is a tool to capture and analyze privacy-enhanced business process models to characterize and quantify to what extent the outputs of a process leak information about its inputs. Pleak incorporates an extensible set of analysis plugins, which enable users to inspect potential leakages at multiple levels of detail.
Pleak is a tool to capture and analyze privacy-enhanced business process models to characterize and quantify to what extent the outputs of a process leak information about its inputs. Pleak incorporates an extensible set of analysis plugins, which enable users to inspect potential leakages at multiple levels of detail.
△ Less
Submitted 13 February, 2019;
originally announced February 2019.
-
Dynamic Role Binding in Blockchain-Based Collaborative Business Processes
Authors:
Orlenys López-Pintado,
Marlon Dumas,
Luciano García-Bañuelos,
Ingo Weber
Abstract:
Blockchain technology enables the execution of collaborative business processes involving mutually untrusted parties. Existing platforms allow such processes to be modeled using high-level notations and compiled into smart contracts that can be deployed on blockchain platforms. However, these platforms brush aside the question of who is allowed to execute which tasks in the process, either by defe…
▽ More
Blockchain technology enables the execution of collaborative business processes involving mutually untrusted parties. Existing platforms allow such processes to be modeled using high-level notations and compiled into smart contracts that can be deployed on blockchain platforms. However, these platforms brush aside the question of who is allowed to execute which tasks in the process, either by deferring the question altogether or by adopting a static approach where all actors are bound to roles upon process instantiation. Yet, a key advantage of blockchains is their ability to support dynamic sets of actors. This paper presents a model for dynamic binding of actors to roles in collaborative processes and an associated binding policy specification language. The proposed language is endowed with a Petri net semantics, thus enabling policy consistency verification. The paper also outlines an approach to compile policy specifications into smart contracts for enforcement. An experimental evaluation shows that the cost of policy enforcement increases linearly with the number of roles and constraints.
△ Less
Submitted 7 December, 2018;
originally announced December 2018.
-
CATERPILLAR: A Business Process Execution Engine on the Ethereum Blockchain
Authors:
Orlenys López-Pintado,
Luciano García-Bañuelos,
Marlon Dumas,
Ingo Weber,
Alex Ponomarev
Abstract:
Blockchain platforms, such as Ethereum, allow a set of actors to maintain a ledger of transactions without relying on a central authority and to deploy scripts, called smart contracts, that are executed whenever certain transactions occur. These features can be used as basic building blocks for executing collaborative business processes between mutually untrusting parties. However, implementing bu…
▽ More
Blockchain platforms, such as Ethereum, allow a set of actors to maintain a ledger of transactions without relying on a central authority and to deploy scripts, called smart contracts, that are executed whenever certain transactions occur. These features can be used as basic building blocks for executing collaborative business processes between mutually untrusting parties. However, implementing business processes using the low-level primitives provided by blockchain platforms is cumbersome and error-prone. In contrast, established business process management systems, such as those based on the standard Business Process Model and Notation (BPMN), provide convenient abstractions for rapid development of process-oriented applications. This article demonstrates how to combine the advantages of a business process management system with those of a blockchain platform. The article introduces a blockchain-based BPMN execution engine, namely Caterpillar. Like any BPMN execution engine, Caterpillar supports the creation of instances of a process model and allows users to monitor the state of process instances and to execute tasks thereof. The specificity of Caterpillar is that the state of each process instance is maintained on the (Ethereum) blockchain and the workflow routing is performed by smart contracts generated by a BPMN-to-Solidity compiler. The Caterpillar compiler supports a large array of BPMN constructs, including subprocesses, multi-instances activities and event handlers. The paper describes the architecture of Caterpillar, and the interfaces it provides to support the monitoring of process instances, the allocation and execution of work items, and the execution of service tasks.
△ Less
Submitted 22 April, 2019; v1 submitted 10 July, 2018;
originally announced August 2018.
-
Blockchain and Principles of Business Process Re-Engineering for Process Innovation
Authors:
Fredrik Milani,
Luciano Garcia-Banuelos
Abstract:
Blockchain has emerged as one of the most promising and revolutionary technologies in the past years. Companies are exploring implementation of use cases in hope of significant gains in efficiencies. However, to achieve the impact hoped for, it is not sufficient to merely replace existing technologies. The current business processes must also be redesigned and innovated to enable realization of ho…
▽ More
Blockchain has emerged as one of the most promising and revolutionary technologies in the past years. Companies are exploring implementation of use cases in hope of significant gains in efficiencies. However, to achieve the impact hoped for, it is not sufficient to merely replace existing technologies. The current business processes must also be redesigned and innovated to enable realization of hoped for benefits. This conceptual paper provides a theoretical contribution on how blockchain technology and smart contracts potentially can, within the framework of the seven principles of business process re-engineering (BPR), enable process innovations. In this paper, we analyze the BPR principles in light of their applicability to blockchain-based solutions. We find these principles to be applicable and helpful in understanding how blockchain technology could enable transformational redesign of current processes. However, the viewpoint taken, should be expanded from intra- to inter-organizational processes operating within an ecosystem of separate organizational entities. In such a blockchain powered ecosystem, smart contracts take on a pivotal role, both as repositories of data and executioner of activities.
△ Less
Submitted 8 June, 2018;
originally announced June 2018.
-
Blockchains for Business Process Management - Challenges and Opportunities
Authors:
Jan Mendling,
Ingo Weber,
Wil van der Aalst,
Jan vom Brocke,
Cristina Cabanillas,
Florian Daniel,
Soren Debois,
Claudio Di Ciccio,
Marlon Dumas,
Schahram Dustdar,
Avigdor Gal,
Luciano Garcia-Banuelos,
Guido Governatori,
Richard Hull,
Marcello La Rosa,
Henrik Leopold,
Frank Leymann,
Jan Recker,
Manfred Reichert,
Hajo A. Reijers,
Stefanie Rinderle-Ma,
Andreas Rogge-Solti,
Michael Rosemann,
Stefan Schulte,
Munindar P. Singh
, et al. (7 additional authors not shown)
Abstract:
Blockchain technology promises a sizable potential for executing inter-organizational business processes without requiring a central party serving as a single point of trust (and failure). This paper analyzes its impact on business process management (BPM). We structure the discussion using two BPM frameworks, namely the six BPM core capabilities and the BPM lifecycle. This paper provides research…
▽ More
Blockchain technology promises a sizable potential for executing inter-organizational business processes without requiring a central party serving as a single point of trust (and failure). This paper analyzes its impact on business process management (BPM). We structure the discussion using two BPM frameworks, namely the six BPM core capabilities and the BPM lifecycle. This paper provides research directions for investigating the application of blockchain technology to BPM.
△ Less
Submitted 31 January, 2018; v1 submitted 11 April, 2017;
originally announced April 2017.
-
Optimized Execution of Business Processes on Blockchain
Authors:
Luciano García-Bañuelos,
Alexander Ponomarev,
Marlon Dumas,
Ingo Weber
Abstract:
Blockchain technology enables the execution of collaborative business processes involving untrusted parties without requiring a central authority. Specifically, a process model comprising tasks performed by multiple parties can be coordinated via smart contracts operating on the blockchain. The consensus mechanism governing the blockchain thereby guarantees that the process model is followed by ea…
▽ More
Blockchain technology enables the execution of collaborative business processes involving untrusted parties without requiring a central authority. Specifically, a process model comprising tasks performed by multiple parties can be coordinated via smart contracts operating on the blockchain. The consensus mechanism governing the blockchain thereby guarantees that the process model is followed by each party. However, the cost required for blockchain use is highly dependent on the volume of data recorded and the frequency of data updates by smart contracts. This paper proposes an optimized method for executing business processes on top of commodity blockchain technology. The paper presents a method for compiling a process model into a smart contract that encodes the preconditions for executing each task in the process using a space-optimized data structure. The method is empirically compared to a previously proposed baseline by replaying execution logs, including one from a real-life business process, and measuring resource consumption.
△ Less
Submitted 9 December, 2016;
originally announced December 2016.
-
Reduction of Event Structures under History Preserving Bisimulation
Authors:
Abel Armas-Cervantes,
Paolo Baldan,
Luciano Garcia-Bañuelos
Abstract:
Event structures represent concurrent processes in terms of events and dependencies between events modelling behavioural relations like causality and conflict. Since the introduction of prime event structures, many variants of event structures have been proposed with different behavioural relations and, hence, with differences in their expressive power. One of the possible benefits of using a more…
▽ More
Event structures represent concurrent processes in terms of events and dependencies between events modelling behavioural relations like causality and conflict. Since the introduction of prime event structures, many variants of event structures have been proposed with different behavioural relations and, hence, with differences in their expressive power. One of the possible benefits of using a more expressive event structure is that of having a more compact representation for the same behaviour when considering the number of events used in a prime event structure. Therefore, this article addresses the problem of reducing the size of an event structure while preserving behaviour under a well-known notion of equivalence, namely history preserving bisimulation. In particular, we investigate this problem on two generalisations of the prime event structures. The first one, known as asymmetric event structure, relies on a asymmetric form of the conflict relation. The second one, known as flow event structure, supports a form of disjunctive causality. More specifically, we describe the conditions under which a set of events in an event structure can be folded into a single event while preserving the original behaviour. The successive application of this folding operation leads to a minimal size event structure. However, the order on which the folding operation is applied may lead to different minimal size event structures. The latter has a negative implication on the potential use of a minimal size event structure as a canonical representation for behaviour.
△ Less
Submitted 30 June, 2014; v1 submitted 27 March, 2014;
originally announced March 2014.
-
Maximal Structuring of Acyclic Process Models
Authors:
Artem Polyvyanyy,
Luciano García-Bañuelos,
Dirk Fahland,
Mathias Weske
Abstract:
This paper contributes to the solution of the problem of transforming a process model with an arbitrary topology into an equivalent structured process model. In particular, this paper addresses the subclass of process models that have no equivalent well-structured representation but which, nevertheless, can be partially structured into their maximally-structured representation. The structuring is…
▽ More
This paper contributes to the solution of the problem of transforming a process model with an arbitrary topology into an equivalent structured process model. In particular, this paper addresses the subclass of process models that have no equivalent well-structured representation but which, nevertheless, can be partially structured into their maximally-structured representation. The structuring is performed under a behavioral equivalence notion that preserves observed concurrency of tasks in equivalent process models. The paper gives a full characterization of the subclass of acyclic process models that have no equivalent well-structured representation but do have an equivalent maximally-structured one, as well as proposes a complete structuring method.
△ Less
Submitted 11 August, 2011;
originally announced August 2011.
