-
PhilHumans: Benchmarking Machine Learning for Personal Health
Authors:
Vadim Liventsev,
Vivek Kumar,
Allmin Pradhap Singh Susaiyah,
Zixiu Wu,
Ivan Rodin,
Asfand Yaar,
Simone Balloccu,
Marharyta Beraziuk,
Sebastiano Battiato,
Giovanni Maria Farinella,
Aki Härmä,
Rim Helaoui,
Milan Petkovic,
Diego Reforgiato Recupero,
Ehud Reiter,
Daniele Riboni,
Raymond Sterling
Abstract:
The use of machine learning in Healthcare has the potential to improve patient outcomes as well as broaden the reach and affordability of Healthcare. The history of other application areas indicates that strong benchmarks are essential for the development of intelligent systems. We present Personal Health Interfaces Leveraging HUman-MAchine Natural interactions (PhilHumans), a holistic suite of be…
▽ More
The use of machine learning in Healthcare has the potential to improve patient outcomes as well as broaden the reach and affordability of Healthcare. The history of other application areas indicates that strong benchmarks are essential for the development of intelligent systems. We present Personal Health Interfaces Leveraging HUman-MAchine Natural interactions (PhilHumans), a holistic suite of benchmarks for machine learning across different Healthcare settings - talk therapy, diet coaching, emergency care, intensive care, obstetric sonography - as well as different learning settings, such as action anticipation, timeseries modeling, insight mining, language modeling, computer vision, reinforcement learning and program synthesis
△ Less
Submitted 16 May, 2024; v1 submitted 4 May, 2024;
originally announced May 2024.
-
Ask the experts: sourcing high-quality datasets for nutritional counselling through Human-AI collaboration
Authors:
Simone Balloccu,
Ehud Reiter,
Vivek Kumar,
Diego Reforgiato Recupero,
Daniele Riboni
Abstract:
Large Language Models (LLMs), with their flexible generation abilities, can be powerful data sources in domains with few or no available corpora. However, problems like hallucinations and biases limit such applications. In this case study, we pick nutrition counselling, a domain lacking any public resource, and show that high-quality datasets can be gathered by combining LLMs, crowd-workers and nu…
▽ More
Large Language Models (LLMs), with their flexible generation abilities, can be powerful data sources in domains with few or no available corpora. However, problems like hallucinations and biases limit such applications. In this case study, we pick nutrition counselling, a domain lacking any public resource, and show that high-quality datasets can be gathered by combining LLMs, crowd-workers and nutrition experts. We first crowd-source and cluster a novel dataset of diet-related issues, then work with experts to prompt ChatGPT into producing related supportive text. Finally, we let the experts evaluate the safety of the generated text. We release HAI-coaching, the first expert-annotated nutrition counselling dataset containing ~2.4K dietary struggles from crowd workers, and ~97K related supportive texts generated by ChatGPT. Extensive analysis shows that ChatGPT while producing highly fluent and human-like text, also manifests harmful behaviours, especially in sensitive topics like mental health, making it unsuitable for unsupervised use.
△ Less
Submitted 16 January, 2024;
originally announced January 2024.
-
Towards Detecting Need for Empathetic Response in Motivational Interviewing
Authors:
Zixiu Wu,
Rim Helaoui,
Vivek Kumar,
Diego Reforgiato Recupero,
Daniele Riboni
Abstract:
Empathetic response from the therapist is key to the success of clinical psychotherapy, especially motivational interviewing. Previous work on computational modelling of empathy in motivational interviewing has focused on offline, session-level assessment of therapist empathy, where empathy captures all efforts that the therapist makes to understand the client's perspective and convey that underst…
▽ More
Empathetic response from the therapist is key to the success of clinical psychotherapy, especially motivational interviewing. Previous work on computational modelling of empathy in motivational interviewing has focused on offline, session-level assessment of therapist empathy, where empathy captures all efforts that the therapist makes to understand the client's perspective and convey that understanding to the client. In this position paper, we propose a novel task of turn-level detection of client need for empathy. Concretely, we propose to leverage pre-trained language models and empathy-related general conversation corpora in a unique labeller-detector framework, where the labeller automatically annotates a motivational interviewing conversation corpus with empathy labels to train the detector that determines the need for therapist empathy. We also lay out our strategies of extending the detector with additional-input and multi-task setups to improve its detection and explainability.
△ Less
Submitted 20 May, 2021;
originally announced May 2021.
-
TF-IDF vs Word Embeddings for Morbidity Identification in Clinical Notes: An Initial Study
Authors:
Danilo Dessi,
Rim Helaoui,
Vivek Kumar,
Diego Reforgiato Recupero,
Daniele Riboni
Abstract:
Today, we are seeing an ever-increasing number of clinical notes that contain clinical results, images, and textual descriptions of patient's health state. All these data can be analyzed and employed to cater novel services that can help people and domain experts with their common healthcare tasks. However, many technologies such as Deep Learning and tools like Word Embeddings have started to be i…
▽ More
Today, we are seeing an ever-increasing number of clinical notes that contain clinical results, images, and textual descriptions of patient's health state. All these data can be analyzed and employed to cater novel services that can help people and domain experts with their common healthcare tasks. However, many technologies such as Deep Learning and tools like Word Embeddings have started to be investigated only recently, and many challenges remain open when it comes to healthcare domain applications. To address these challenges, we propose the use of Deep Learning and Word Embeddings for identifying sixteen morbidity types within textual descriptions of clinical records. For this purpose, we have used a Deep Learning model based on Bidirectional Long-Short Term Memory (LSTM) layers which can exploit state-of-the-art vector representations of data such as Word Embeddings. We have employed pre-trained Word Embeddings namely GloVe and Word2Vec, and our own Word Embeddings trained on the target domain. Furthermore, we have compared the performances of the deep learning approaches against the traditional tf-idf using Support Vector Machine and Multilayer perceptron (our baselines). From the obtained results it seems that the latter outperforms the combination of Deep Learning approaches using any word embeddings. Our preliminary results indicate that there are specific features that make the dataset biased in favour of traditional machine learning approaches.
△ Less
Submitted 9 June, 2021; v1 submitted 20 May, 2021;
originally announced May 2021.
-
Extended Report: Fine-grained Recognition of Abnormal Behaviors for Early Detection of Mild Cognitive Impairment
Authors:
Daniele Riboni,
Claudio Bettini,
Gabriele Civitarese,
Zaffar Haider Janjua,
Rim Helaoui
Abstract:
According to the World Health Organization, the rate of people aged 60 or more is growing faster than any other age group in almost every country, and this trend is not going to change in a near future. Since senior citizens are at high risk of non communicable diseases requiring long-term care, this trend will challenge the sustainability of the entire health system. Pervasive computing can provi…
▽ More
According to the World Health Organization, the rate of people aged 60 or more is growing faster than any other age group in almost every country, and this trend is not going to change in a near future. Since senior citizens are at high risk of non communicable diseases requiring long-term care, this trend will challenge the sustainability of the entire health system. Pervasive computing can provide innovative methods and tools for early detecting the onset of health issues. In this paper we propose a novel method to detect abnormal behaviors of elderly people living at home. The method relies on medical models, provided by cognitive neuroscience researchers, describing abnormal activity routines that may indicate the onset of early symptoms of mild cognitive impairment. A non-intrusive sensor-based infrastructure acquires low-level data about the interaction of the individual with home appliances and furniture, as well as data from environmental sensors. Based on those data, a novel hybrid statistical-symbolical technique is used to detect the abnormal behaviors of the patient, which are communicated to the medical center. Differently from related works, our method can detect abnormal behaviors at a fine-grained level, thus providing an important tool to support the medical diagnosis. In order to evaluate our method we have developed a prototype of the system and acquired a large dataset of abnormal behaviors carried out in an instrumented smart home. Experimental results show that our technique is able to detect most anomalies while generating a small number of false positives.
△ Less
Submitted 22 January, 2015;
originally announced January 2015.
-
Preserving Privacy in Sequential Data Release against Background Knowledge Attacks
Authors:
Daniele Riboni,
Linda Pareschi,
Claudio Bettini
Abstract:
A large amount of transaction data containing associations between individuals and sensitive information flows everyday into data stores. Examples include web queries, credit card transactions, medical exam records, transit database records. The serial release of these data to partner institutions or data analysis centers is a common situation. In this paper we show that, in most domains, correlat…
▽ More
A large amount of transaction data containing associations between individuals and sensitive information flows everyday into data stores. Examples include web queries, credit card transactions, medical exam records, transit database records. The serial release of these data to partner institutions or data analysis centers is a common situation. In this paper we show that, in most domains, correlations among sensitive values associated to the same individuals in different releases can be easily mined, and used to violate users' privacy by adversaries observing multiple data releases. We provide a formal model for privacy attacks based on this sequential background knowledge, as well as on background knowledge on the probability distribution of sensitive values over different individuals. We show how sequential background knowledge can be actually obtained by an adversary, and used to identify with high confidence the sensitive values associated with an individual. A defense algorithm based on Jensen-Shannon divergence is proposed, and extensive experiments show the superiority of the proposed technique with respect to other applicable solutions. To the best of our knowledge, this is the first work that systematically investigates the role of sequential background knowledge in serial release of transaction data.
△ Less
Submitted 5 October, 2010;
originally announced October 2010.
