-
A Data-driven Neural Network Architecture for Sentiment Analysis
Authors:
Erion Çano,
Maurizio Morisio
Abstract:
The fabulous results of convolution neural networks in image-related tasks, attracted attention of text mining, sentiment analysis and other text analysis researchers. It is however difficult to find enough data for feeding such networks, optimize their parameters, and make the right design choices when constructing network architectures. In this paper we present the creation steps of two big data…
▽ More
The fabulous results of convolution neural networks in image-related tasks, attracted attention of text mining, sentiment analysis and other text analysis researchers. It is however difficult to find enough data for feeding such networks, optimize their parameters, and make the right design choices when constructing network architectures. In this paper we present the creation steps of two big datasets of song emotions. We also explore usage of convolution and max-pooling neural layers on song lyrics, product and movie review text datasets. Three variants of a simple and flexible neural network architecture are also compared. Our intention was to spot any important patterns that can serve as guidelines for parameter optimization of similar models. We also wanted to identify architecture design choices which lead to high performing sentiment analysis models. To this end, we conducted a series of experiments with neural architectures of various configurations. Our results indicate that parallel convolutions of filter lengths up to three are usually enough for capturing relevant text features. Also, max-pooling region size should be adapted to the length of text documents for producing the best feature maps. Top results we got are obtained with feature maps of lengths 6 to 18. An improvement on future neural network models for sentiment analysis, could be generating sentiment polarity prediction of documents using aggregation of predictions on smaller excerpt of the entire text.
△ Less
Submitted 30 June, 2020;
originally announced June 2020.
-
Mood-based On-Car Music Recommendations
Authors:
Erion Çano,
Riccardo Coppola,
Eleonora Gargiulo,
Marco Marengo,
Maurizio Morisio
Abstract:
Driving and music listening are two inseparable everyday activities for millions of people today in the world. Considering the high correlation between music, mood and driving comfort and safety, it makes sense to use appropriate and intelligent music recommendations based on the mood of drivers and songs in the context of car driving. The objective of this paper is to present the project of a con…
▽ More
Driving and music listening are two inseparable everyday activities for millions of people today in the world. Considering the high correlation between music, mood and driving comfort and safety, it makes sense to use appropriate and intelligent music recommendations based on the mood of drivers and songs in the context of car driving. The objective of this paper is to present the project of a contextual mood-based music recommender system capable of regulating the driver's mood and trying to have a positive influence on her driving behaviour. Here we present the proof of concept of the system and describe the techniques and technologies that are part of it. Further possible future improvements on each of the building blocks are also presented.
△ Less
Submitted 25 June, 2020;
originally announced June 2020.
-
Quality of Word Embeddings on Sentiment Analysis Tasks
Authors:
Erion Çano,
Maurizio Morisio
Abstract:
Word embeddings or distributed representations of words are being used in various applications like machine translation, sentiment analysis, topic identification etc. Quality of word embeddings and performance of their applications depends on several factors like training method, corpus size and relevance etc. In this study we compare performance of a dozen of pretrained word embedding models on l…
▽ More
Word embeddings or distributed representations of words are being used in various applications like machine translation, sentiment analysis, topic identification etc. Quality of word embeddings and performance of their applications depends on several factors like training method, corpus size and relevance etc. In this study we compare performance of a dozen of pretrained word embedding models on lyrics sentiment analysis and movie review polarity tasks. According to our results, Twitter Tweets is the best on lyrics sentiment analysis, whereas Google News and Common Crawl are the top performers on movie polarity analysis. Glove trained models slightly outrun those trained with Skipgram. Also, factors like topic relevance and size of corpus significantly impact the quality of the models. When medium or large-sized text sets are available, obtaining word embeddings from same training dataset is usually the best choice.
△ Less
Submitted 6 March, 2020;
originally announced March 2020.
-
Predict your Click-out: Modeling User-Item Interactions and Session Actions in an Ensemble Learning Fashion
Authors:
Andrea Fiandro,
Giorgio Crepaldi,
Diego Monti,
Giuseppe Rizzo,
Maurizio Morisio
Abstract:
This paper describes the solution of the POLINKS team to the RecSys Challenge 2019 that focuses on the task of predicting the last click-out in a session-based interaction. We propose an ensemble approach comprising a matrix factorization for modeling the interaction user-item, and a session-aware learning model implemented with a recurrent neural network. This method appears to be effective in pr…
▽ More
This paper describes the solution of the POLINKS team to the RecSys Challenge 2019 that focuses on the task of predicting the last click-out in a session-based interaction. We propose an ensemble approach comprising a matrix factorization for modeling the interaction user-item, and a session-aware learning model implemented with a recurrent neural network. This method appears to be effective in predicting the last click-out scoring a 0.60277 of Mean Reciprocal Rank on the local test set.
△ Less
Submitted 8 February, 2020;
originally announced February 2020.
-
All You Need is Ratings: A Clustering Approach to Synthetic Rating Datasets Generation
Authors:
Diego Monti,
Giuseppe Rizzo,
Maurizio Morisio
Abstract:
The public availability of collections containing user preferences is of vital importance for performing offline evaluations in the field of recommender systems. However, the number of rating datasets is limited because of the costs required for their creation and the fear of violating the privacy of the users by sharing them. For this reason, numerous research attempts investigated the creation o…
▽ More
The public availability of collections containing user preferences is of vital importance for performing offline evaluations in the field of recommender systems. However, the number of rating datasets is limited because of the costs required for their creation and the fear of violating the privacy of the users by sharing them. For this reason, numerous research attempts investigated the creation of synthetic collections of ratings using generative approaches. Nevertheless, these datasets are usually not reliable enough for conducting an evaluation campaign. In this paper, we propose a method for creating synthetic datasets with a configurable number of users that mimic the characteristics of already existing ones. We empirically validated the proposed approach by exploiting the synthetic datasets for evaluating different recommenders and by comparing the results with the ones obtained using real datasets.
△ Less
Submitted 2 September, 2019;
originally announced September 2019.
-
Word Embeddings for Sentiment Analysis: A Comprehensive Empirical Survey
Authors:
Erion Çano,
Maurizio Morisio
Abstract:
This work investigates the role of factors like training method, training corpus size and thematic relevance of texts in the performance of word embedding features on sentiment analysis of tweets, song lyrics, movie reviews and item reviews. We also explore specific training or post-processing methods that can be used to enhance the performance of word embeddings in certain tasks or domains. Our e…
▽ More
This work investigates the role of factors like training method, training corpus size and thematic relevance of texts in the performance of word embedding features on sentiment analysis of tweets, song lyrics, movie reviews and item reviews. We also explore specific training or post-processing methods that can be used to enhance the performance of word embeddings in certain tasks or domains. Our empirical observations indicate that models trained with multithematic texts that are large and rich in vocabulary are the best in answering syntactic and semantic word analogy questions. We further observe that influence of thematic relevance is stronger on movie and phone reviews, but weaker on tweets and lyrics. These two later domains are more sensitive to corpus size and training method, with Glove outperforming Word2vec. "Injecting" extra intelligence from lexicons or generating sentiment specific word embeddings are two prominent alternatives for increasing performance of word embedding features.
△ Less
Submitted 2 February, 2019;
originally announced February 2019.
-
Hybrid Recommender Systems: A Systematic Literature Review
Authors:
Erion Çano,
Maurizio Morisio
Abstract:
Recommender systems are software tools used to generate and provide suggestions for items and other entities to the users by exploiting various strategies. Hybrid recommender systems combine two or more recommendation strategies in different ways to benefit from their complementary advantages. This systematic literature review presents the state of the art in hybrid recommender systems of the last…
▽ More
Recommender systems are software tools used to generate and provide suggestions for items and other entities to the users by exploiting various strategies. Hybrid recommender systems combine two or more recommendation strategies in different ways to benefit from their complementary advantages. This systematic literature review presents the state of the art in hybrid recommender systems of the last decade. It is the first quantitative review work completely focused in hybrid recommenders. We address the most relevant problems considered and present the associated data mining and recommendation techniques used to overcome them. We also explore the hybridization classes each hybrid recommender belongs to, the application domains, the evaluation process and proposed future research directions. Based on our findings, most of the studies combine collaborative filtering with another technique often in a weighted way. Also cold-start and data sparsity are the two traditional and top problems being addressed in 23 and 22 studies each, while movies and movie datasets are still widely used by most of the authors. As most of the studies are evaluated by comparisons with similar methods using accuracy metrics, providing more credible and user oriented evaluations remains a typical challenge. Besides this, newer challenges were also identified such as responding to the variation of user context, evolving user tastes or providing cross-domain recommendations. Being a hot topic, hybrid recommenders represent a good basis with which to respond accordingly by exploring newer opportunities such as contextualizing recommendations, involving parallel hybrid algorithms, processing larger datasets, etc.
△ Less
Submitted 12 January, 2019;
originally announced January 2019.
-
A Software Ecosystem Reshaped by a Paradigm Shift: the CSI-Piemonte Case
Authors:
Federico Tomassetti,
Marco Torchiano,
Mauro Antonaci,
Paolo Arvati,
Maurizio Morisio
Abstract:
Context: Changes in the software development paradigm, when operated by entities with a pivotal role, have the power to affect a number of groups and entities in their sphere of influence, changing both their working habits and relations.
Objective: In this paper we present the organizational changes occurred in a software ecosystem as consequence of a technological change. In particular we exam…
▽ More
Context: Changes in the software development paradigm, when operated by entities with a pivotal role, have the power to affect a number of groups and entities in their sphere of influence, changing both their working habits and relations.
Objective: In this paper we present the organizational changes occurred in a software ecosystem as consequence of a technological change. In particular we examine the evolution of an MDD solution and the changing roles of the company promoting it, the public administrations and the sub-contractors.
Method: The paper focuses on a single case study that encompasses the six years long evolution of a Model-driven development solution, starting from its conception until is recent open-source release, across five distinct phases. The history was analyzed jointly by software engineering academics and industrial managers directly involved in the case study.
Results: A report of the ecosystem evolution from an idiographic perspective is reported. An analysis of the history allowed an abstraction that led to the identification of several distinct ecosystem evolution motifs.
Conclusion: The motifs represent a set of key process areas for the evolution of a software ecosystem. They are potentially generalizable to other similar ecosystems. As such, they can be used by researchers to evaluate existing in-progress case studies, and by practitioners as a set of guidelines.
△ Less
Submitted 18 December, 2018;
originally announced December 2018.
-
Semantic Trails of City Explorations: How Do We Live a City
Authors:
Diego Monti,
Enrico Palumbo,
Giuseppe Rizzo,
Raphaël Troncy,
Thibault Ehrhart,
Maurizio Morisio
Abstract:
The knowledge of city exploration trails of people is in short supply because of the complexity in defining meaningful trails representative of individual behaviours and in the access to actionable data. Existing datasets have only recorded isolated check-ins of activities featured by opaque venue types. In this paper, we fill the gaps in defining what is a semantic trail of city exploration and h…
▽ More
The knowledge of city exploration trails of people is in short supply because of the complexity in defining meaningful trails representative of individual behaviours and in the access to actionable data. Existing datasets have only recorded isolated check-ins of activities featured by opaque venue types. In this paper, we fill the gaps in defining what is a semantic trail of city exploration and how it can be generated by integrating different data sources. Furthermore, we publicly release two datasets holding millions of semantic trails each and we discuss their most salient characteristics. We finally present an application using these datasets to build a recommender system meant to guide tourists while exploring a city.
△ Less
Submitted 30 December, 2019; v1 submitted 11 December, 2018;
originally announced December 2018.
-
A Distributed and Accountable Approach to Offline Recommender Systems Evaluation
Authors:
Diego Monti,
Giuseppe Rizzo,
Maurizio Morisio
Abstract:
Different software tools have been developed with the purpose of performing offline evaluations of recommender systems. However, the results obtained with these tools may be not directly comparable because of subtle differences in the experimental protocols and metrics. Furthermore, it is difficult to analyze in the same experimental conditions several algorithms without disclosing their implement…
▽ More
Different software tools have been developed with the purpose of performing offline evaluations of recommender systems. However, the results obtained with these tools may be not directly comparable because of subtle differences in the experimental protocols and metrics. Furthermore, it is difficult to analyze in the same experimental conditions several algorithms without disclosing their implementation details. For these reasons, we introduce RecLab, an open source software for evaluating recommender systems in a distributed fashion. By relying on consolidated web protocols, we created RESTful APIs for training and querying recommenders remotely. In this way, it is possible to easily integrate into the same toolkit algorithms realized with different technologies. In details, the experimenter can perform an evaluation by simply visiting a web interface provided by RecLab. The framework will then interact with all the selected recommenders and it will compute and display a comprehensive set of measures, each representing a different metric. The results of all experiments are permanently stored and publicly available in order to support accountability and comparative analyses.
△ Less
Submitted 11 October, 2018;
originally announced October 2018.
-
Sequeval: A Framework to Assess and Benchmark Sequence-based Recommender Systems
Authors:
Diego Monti,
Enrico Palumbo,
Giuseppe Rizzo,
Maurizio Morisio
Abstract:
In this paper, we present sequeval, a software tool capable of performing the offline evaluation of a recommender system designed to suggest a sequence of items. A sequence-based recommender is trained considering the sequences already available in the system and its purpose is to generate a personalized sequence starting from an initial seed. This tool automatically evaluates the sequence-based r…
▽ More
In this paper, we present sequeval, a software tool capable of performing the offline evaluation of a recommender system designed to suggest a sequence of items. A sequence-based recommender is trained considering the sequences already available in the system and its purpose is to generate a personalized sequence starting from an initial seed. This tool automatically evaluates the sequence-based recommender considering a comprehensive set of eight different metrics adapted to the sequential scenario. sequeval has been developed following the best practices of software extensibility. For this reason, it is possible to easily integrate and evaluate novel recommendation techniques. sequeval is publicly available as an open source tool and it aims to become a focal point for the community to assess sequence-based recommender systems.
△ Less
Submitted 29 November, 2018; v1 submitted 11 October, 2018;
originally announced October 2018.
-
Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility
Authors:
Riccardo Coppola,
Maurizio Morisio,
Marco Torchiano
Abstract:
Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e. GUI test classes failing due to minor modifications in the GUI, without the application functionalities being altered.
Aims. The objective of our study is to ex…
▽ More
Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e. GUI test classes failing due to minor modifications in the GUI, without the application functionalities being altered.
Aims. The objective of our study is to examine the diffusion of GUI testing on Android, and the amount of changes required to keep test classes up to date, and in particular the changes due to GUI test fragility. We define metrics to characterize the modifications and evolution of test classes and test methods, and proxies to estimate fragility-induced changes.
Method. To perform our experiments, we selected six widely used open-source tools for scripted GUI testing of mobile applications previously described in the literature. We have mined the repositories on GitHub that used those tools, and computed our set of metrics.
Results. We found that none of the considered GUI testing frameworks achieved a major diffusion among the open-source Android projects available on GitHub. For projects with GUI tests, we found that test suites have to be modified often, specifically 5\%-10\% of developers' modified LOCs belong to tests, and that a relevant portion (60\% on average) of such modifications are induced by fragility.
Conclusions. Fragility of GUI test classes constitute a relevant concern, possibly being an obstacle for developers to adopt automated scripted GUI tests. This first evaluation and measure of fragility of Android scripted GUI testing can constitute a benchmark for developers, and the basis for the definition of a taxonomy of fragility causes, and actionable guidelines to mitigate the issue.
△ Less
Submitted 9 November, 2017;
originally announced November 2017.
-
Content Recommendation through Semantic Annotation of User Reviews and Linked Data - An Extended Technical Report
Authors:
Iacopo Vagliano,
Diego Monti,
Ansgar Scherp,
Maurizio Morisio
Abstract:
Nowadays, most recommender systems exploit user-provided ratings to infer their preferences. However, the growing popularity of social and e-commerce websites has encouraged users to also share comments and opinions through textual reviews. In this paper, we introduce a new recommendation approach which exploits the semantic annotation of user reviews to extract useful and non-trivial information…
▽ More
Nowadays, most recommender systems exploit user-provided ratings to infer their preferences. However, the growing popularity of social and e-commerce websites has encouraged users to also share comments and opinions through textual reviews. In this paper, we introduce a new recommendation approach which exploits the semantic annotation of user reviews to extract useful and non-trivial information about the items to recommend. It also relies on the knowledge freely available in the Web of Data, notably in DBpedia and Wikidata, to discover other resources connected with the annotated entities. We evaluated our approach in three domains, using both DBpedia and Wikidata. The results showed that our solution provides a better ranking than another recommendation method based on the Web of Data, while it improves in novelty with respect to traditional techniques based on ratings. Additionally, our method achieved a better performance with Wikidata than DBpedia.
△ Less
Submitted 28 October, 2017; v1 submitted 28 September, 2017;
originally announced September 2017.
