How Far are we from Data Mining Democratisation? A Systematic Review
Authors:
Alfonso de la Vega,
Diego García-Saiz,
Marta Zorrilla,
Pablo Sánchez
Abstract:
Context: Data mining techniques have demonstrated to be a powerful technique for discovering insights hidden in data from a domain. However, these techniques demand very specialised skills. People willing to analyse data often lack these skills, so they must rely on data scientists, which hinders data mining democratisation. Different approaches have appeared in the last years to address this issu…
▽ More
Context: Data mining techniques have demonstrated to be a powerful technique for discovering insights hidden in data from a domain. However, these techniques demand very specialised skills. People willing to analyse data often lack these skills, so they must rely on data scientists, which hinders data mining democratisation. Different approaches have appeared in the last years to address this issue.
Objective: Analyse the state of the art to know how far are we from an effective data mining democratisation, what has already been accomplished, and what should be done in the upcoming years.
Method: We performed a state-of-the-art review following a systematic and objective procedure, which included works both from the academia and the industry. The reviewed works were grouped in four categories. Each category was then evaluated in detail using a well-defined evaluation criteria to identify its strengths and weaknesses.
Results: Around 700 works were initially considered, from which 43 were finally selected for a more in-depth analysis. Only two out of the four identified categories provide effective solutions to data mining democratisation. From these two categories, one always requires a minimum intervention of a data scientist, whereas the other one does not provide support for all the stages of the data mining process, and might exhibit accuracy problems in some contexts.
Conclusion: In all analysed approaches, a data scientist is still required to perform some steps of the analysis process. Moreover, automated approaches that do not require data scientists for some steps expose some problems in other quality attributes, such as accuracy. Therefore, although existent work shows some promising initial steps, we are still far from data mining democratisation.
△ Less
Submitted 20 March, 2019;
originally announced March 2019.
Closed-set-based Discovery of Bases of Association Rules
Authors:
José L. Balcázar,
Diego García-Saiz,
Domingo Gómez-Pérez,
Cristina Tîrnăucă
Abstract:
The output of an association rule miner is often huge in practice. This is why several concise lossless representations have been proposed, such as the "essential" or "representative" rules. We revisit the algorithm given by Kryszkiewicz (Int. Symp. Intelligent Data Analysis 2001, Springer-Verlag LNCS 2189, 350-359) for mining representative rules. We show that its output is sometimes incomplete…
▽ More
The output of an association rule miner is often huge in practice. This is why several concise lossless representations have been proposed, such as the "essential" or "representative" rules. We revisit the algorithm given by Kryszkiewicz (Int. Symp. Intelligent Data Analysis 2001, Springer-Verlag LNCS 2189, 350-359) for mining representative rules. We show that its output is sometimes incomplete, due to an oversight in its mathematical validation. We propose alternative complete generators and we extend the approach to an existing closure-aware basis similar to, and often smaller than, the representative rules, namely the basis B*.
△ Less
Submitted 24 March, 2011; v1 submitted 3 December, 2010;
originally announced December 2010.
