-
Text-to-Image Generation for Vocabulary Learning Using the Keyword Method
Authors:
Nuwan T. Attygalle,
Matjaž Kljun,
Aaron Quigley,
Klen čOpič Pucihar,
Jens Grubert,
Verena Biener,
Luis A. Leiva,
Juri Yoneyama,
Alice Toniolo,
Angela Miguel,
Hirokazu Kato,
Maheshya Weerasinghe
Abstract:
The 'keyword method' is an effective technique for learning vocabulary of a foreign language. It involves creating a memorable visual link between what a word means and what its pronunciation in a foreign language sounds like in the learner's native language. However, these memorable visual links remain implicit in the people's mind and are not easy to remember for a large set of words. To enhance…
▽ More
The 'keyword method' is an effective technique for learning vocabulary of a foreign language. It involves creating a memorable visual link between what a word means and what its pronunciation in a foreign language sounds like in the learner's native language. However, these memorable visual links remain implicit in the people's mind and are not easy to remember for a large set of words. To enhance the memorisation and recall of the vocabulary, we developed an application that combines the keyword method with text-to-image generators to externalise the memorable visual links into visuals. These visuals represent additional stimuli during the memorisation process. To explore the effectiveness of this approach we first run a pilot study to investigate how difficult it is to externalise the descriptions of mental visualisations of memorable links, by asking participants to write them down. We used these descriptions as prompts for text-to-image generator (DALL-E2) to convert them into images and asked participants to select their favourites. Next, we compared different text-to-image generators (DALL-E2, Midjourney, Stable and Latent Diffusion) to evaluate the perceived quality of the generated images by each. Despite heterogeneous results, participants mostly preferred images generated by DALL-E2, which was used also for the final study. In this study, we investigated whether providing such images enhances the retention of vocabulary being learned, compared to the keyword method only. Our results indicate that people did not encounter difficulties describing their visualisations of memorable links and that providing corresponding images significantly improves memory retention.
△ Less
Submitted 28 January, 2025;
originally announced January 2025.
-
Retzzles: Do Jigsaw Puzzle Actions on Interactive Display Maps Increase the Retention of Map Information?
Authors:
Nikola Kovačević,
Jordan Aiko Deja,
Maheshya Weerasinghe,
Klen Čopič Pucihar,
Matjaž Kljun
Abstract:
While maps provide upfront content, this might not always be the most effective way for users to remember information. With the proliferation of interactive displays for tourists and visitors in public spaces, we can create a more playful user experience with maps than just exploring them. Adding interactions with the map could also help users retain more information as they use them. In this pape…
▽ More
While maps provide upfront content, this might not always be the most effective way for users to remember information. With the proliferation of interactive displays for tourists and visitors in public spaces, we can create a more playful user experience with maps than just exploring them. Adding interactions with the map could also help users retain more information as they use them. In this paper, we investigated whether completing a jigsaw puzzle of a map supports users in retaining more information about a specific map. The results of a between-subject study with a sample of n=28 indicate that additional interaction helped improve mean scores of textual and spatial recall but not visual recall. However, the results are not statistically significant, and the topic is subject to further investigation. Our findings contribute to discussions on using interactive touchscreen displays in similar learning scenarios involving memory retention.
△ Less
Submitted 9 February, 2024;
originally announced February 2024.
-
'The Taurus': Cattle Breeds & Diseases Identification Mobile Application using Machine Learning
Authors:
R. M. D. S. M. Chandrarathna,
T. W. M. S. A. Weerasinghe,
N. S. Madhuranga,
T. M. L. S. Thennakoon,
Anjalie Gamage,
Erandika Lakmali
Abstract:
Dairy farming plays an important role in agriculture for thousands of years not only in Sri Lanka but also in so many other countries. When it comes to dairy farming cattle is an indispensable animal. According to the literature surveys almost 3.9 million cattle and calves die in a year due to different types of diseases. The causes of diseases are mainly bacteria, parasites, fungi, chemical poiso…
▽ More
Dairy farming plays an important role in agriculture for thousands of years not only in Sri Lanka but also in so many other countries. When it comes to dairy farming cattle is an indispensable animal. According to the literature surveys almost 3.9 million cattle and calves die in a year due to different types of diseases. The causes of diseases are mainly bacteria, parasites, fungi, chemical poisons and etc. Infectious diseases can be a greatest threat to livestock health. The mortality rate of cattle causes a huge impact on social, economic and environmental damage. In order to decrease this negative impact, the proposal implements a cross-platform mobile application to easily analyze and identify the diseases which cattle suffer from and give them a solution and also to identify the cattle breeds. The mobile application is designed to identify the breeds by analyzing the images of the cattle and identify diseases after analyzing the videos and the images of affected areas. Then make a model to identify the weight and the age of a particular cow and suggest the best dose of the medicine to the identified disease. This will be a huge advantage to farmers as well as to dairy industry. The name of the proposed mobile application is 'The Taurus' and this paper address the selected machine learning and image processing models and the approaches taken to identify the diseases, breeds and suggest the prevention methods and medicine to the identified disease.
△ Less
Submitted 20 February, 2023;
originally announced February 2023.
-
VocabulARy replicated: comparing teenagers to young adults
Authors:
Maheshya Weerasinghe,
Verena Biener,
Jens Grubert,
Jordan Aiko Deja,
Nuwan T. Attygalle,
Karolina Trajkovska,
Matjaž Kljun,
Klen Čopič Pucihar
Abstract:
A critical component of user studies is gaining access to a representative sample of the population researches intend to investigate. Nevertheless, the vast majority of human-computer interaction (HCI)studies, including augmented reality (AR) studies, rely on convenience sampling. The outcomes of these studies are often based on results obtained from university students aged between 19 and 26 year…
▽ More
A critical component of user studies is gaining access to a representative sample of the population researches intend to investigate. Nevertheless, the vast majority of human-computer interaction (HCI)studies, including augmented reality (AR) studies, rely on convenience sampling. The outcomes of these studies are often based on results obtained from university students aged between 19 and 26 years. In order to investigate how the results from one of our studies are affected by convenience sampling, we replicated the AR-supported language learning study called VocabulARy with 24 teenagers, aged between 14 and 19 years. The results verified most of the outcomes from the original study. In addition, it also revealed that teenagers found learning significantly less mentally demanding compared to young adults, and completed the study in a significantly shorter time. All this at no cost to learning outcomes.
△ Less
Submitted 31 October, 2022;
originally announced October 2022.
-
VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations
Authors:
Maheshya Weerasinghe,
Verena Biener,
Jens Grubert,
Aaron J Quigley,
Alice Toniolo,
Klen Čopič Pucihar,
Matjaž Kljun
Abstract:
Learning vocabulary in a primary or secondary language is enhanced when we encounter words in context. This context can be afforded by the place or activity we are engaged with. Existing learning environments include formal learning, mnemonics, flashcards, use of a dictionary or thesaurus, all leading to practice with new words in context. In this work, we propose an enhancement to the language le…
▽ More
Learning vocabulary in a primary or secondary language is enhanced when we encounter words in context. This context can be afforded by the place or activity we are engaged with. Existing learning environments include formal learning, mnemonics, flashcards, use of a dictionary or thesaurus, all leading to practice with new words in context. In this work, we propose an enhancement to the language learning process by providing the user with words and learning tools in context, with VocabulARy. VocabulARy visually annotates objects in AR, in the user's surroundings, with the corresponding English (first language) and Japanese (second language) words to enhance the language learning process. In addition to the written and audio description of each word, we also present the user with a keyword and its visualisation to enhance memory retention. We evaluate our prototype by comparing it to an alternate AR system that does not show an additional visualisation of the keyword, and, also, we compare it to two non-AR systems on a tablet, one with and one without visualising the keyword. Our results indicate that AR outperforms the tablet system regarding immediate recall, mental effort and task completion time. Additionally, the visualisation approach scored significantly higher than showing only the written keyword with respect to immediate and delayed recall and learning efficiency, mental effort and task-completion time.
△ Less
Submitted 2 July, 2022;
originally announced July 2022.
-
Arigatō: Effects of Adaptive Guidance on Engagement and Performance in Augmented Reality Learning Environments
Authors:
Maheshya Weerasinghe,
Aaron Quigley,
Klen Čopič Pucihar,
Alice Toniolo,
Angela Miguel,
Matjaž Kljun
Abstract:
Experiential learning (ExL) is the process of learning through experience or more specifically "learning through reflection on doing". In this paper, we propose a simulation of these experiences, in Augmented Reality (AR), addressing the problem of language learning. Such systems provide an excellent setting to support "adaptive guidance", in a digital form, within a real environment. Adaptive gui…
▽ More
Experiential learning (ExL) is the process of learning through experience or more specifically "learning through reflection on doing". In this paper, we propose a simulation of these experiences, in Augmented Reality (AR), addressing the problem of language learning. Such systems provide an excellent setting to support "adaptive guidance", in a digital form, within a real environment. Adaptive guidance allows the instructions and learning content to be customised for the individual learner, thus creating a unique learning experience. We developed an adaptive guidance AR system for language learning, we call Arigatō (Augmented Reality Instructional Guidance & Tailored Omniverse), which offers immediate assistance, resources specific to the learner's needs, manipulation of these resources, and relevant feedback. Considering guidance, we employ this prototype to investigate the effect of the amount of guidance (fixed vs. adaptive-amount) and the type of guidance (fixed vs. adaptive-associations) on the engagement and consequently the learning outcomes of language learning in an AR environment. The results for the amount of guidance show that compared to the adaptive-amount, the fixed-amount of guidance group scored better in the immediate and delayed (after 7 days) recall tests. However, this group also invested a significantly higher mental effort to complete the task. The results for the type of guidance show that the adaptive-associations group outperforms the fixed-associations group in the immediate, delayed (after 7 days) recall tests, and learning efficiency. The adaptive-associations group also showed significantly lower mental effort and spent less time to complete the task.
△ Less
Submitted 2 July, 2022;
originally announced July 2022.
-
Quantifying the Effects of Working in VR for One Week
Authors:
Verena Biener,
Snehanjali Kalamkar,
Negar Nouri,
Eyal Ofek,
Michel Pahud,
John J. Dudley,
Jinghui Hu,
Per Ola Kristensson,
Maheshya Weerasinghe,
Klen Čopič Pucihar,
Matjaž Kljun,
Stephan Streuber,
Jens Grubert
Abstract:
Virtual Reality (VR) provides new possibilities for modern knowledge work. However, the potential advantages of virtual work environments can only be used if it is feasible to work in them for an extended period of time. Until now, there are limited studies of long-term effects when working in VR. This paper addresses the need for understanding such long-term effects. Specifically, we report on a…
▽ More
Virtual Reality (VR) provides new possibilities for modern knowledge work. However, the potential advantages of virtual work environments can only be used if it is feasible to work in them for an extended period of time. Until now, there are limited studies of long-term effects when working in VR. This paper addresses the need for understanding such long-term effects. Specifically, we report on a comparative study (n=16), in which participants were working in VR for an entire week -- for five days, eight hours each day -- as well as in a baseline physical desktop environment. This study aims to quantify the effects of exchanging a desktop-based work environment with a VR-based environment. Hence, during this study, we do not present the participants with the best possible VR system but rather a setup delivering a comparable experience to working in the physical desktop environment. The study reveals that, as expected, VR results in significantly worse ratings across most measures. Among other results, we found concerning levels of simulator sickness, below average usability ratings and two participants dropped out on the first day using VR, due to migraine, nausea and anxiety. Nevertheless, there is some indication that participants gradually overcame negative first impressions and initial discomfort. Overall, this study helps lay the groundwork for subsequent research, by clearly highlighting current shortcomings and identifying opportunities for improving the experience of working in VR.
△ Less
Submitted 8 June, 2022; v1 submitted 7 June, 2022;
originally announced June 2022.
-
An ICT based Solution for Virtual Garment Fitting for Online Market Place: A Review of Related Literature
Authors:
Hashini Gunatilake,
Dulaji Hidellaarachchi,
Sandra Perera,
Damitha Sandaruwan,
Maheshya Weerasinghe
Abstract:
In this paper, we describe various technologies that are being used in virtual garment fitting and simulation. There, we have focused about the usage of anthropometry in clothing industry and avatar generation of virtual garment fitting. Most commonly used technologies for avatar generation in virtual environment have been discussed in this paper such as generic body model and laser scanning. More…
▽ More
In this paper, we describe various technologies that are being used in virtual garment fitting and simulation. There, we have focused about the usage of anthropometry in clothing industry and avatar generation of virtual garment fitting. Most commonly used technologies for avatar generation in virtual environment have been discussed in this paper such as generic body model and laser scanning. Moreover, this paper includes the real-time tracking technologies used in virtual garment fitting like markers and depth cameras in various related researches as well as how the virtual cloth generation and simulation carried out in the related researches. Apart from these, virtual clothing methods such as geometrical, physical and hybrid based models were also discussed in this paper. As ease allowance has a major impact on virtual cloth fitting, it is also considered in this paper related to similar researches. Within this paper, all the above mentioned areas were described thoroughly while stating the existing gap of the virtual garment fitting in online marketplaces.
△ Less
Submitted 31 March, 2022;
originally announced April 2022.
-
Ensemble plasticity and network adaptability in SNNs
Authors:
Mahima Milinda Alwis Weerasinghe,
David Parry,
Grace Wang,
Jacqueline Whalley
Abstract:
Artificial Spiking Neural Networks (ASNNs) promise greater information processing efficiency because of discrete event-based (i.e., spike) computation. Several Machine Learning (ML) applications use biologically inspired plasticity mechanisms as unsupervised learning techniques to increase the robustness of ASNNs while preserving efficiency. Spike Time Dependent Plasticity (STDP) and Intrinsic Pla…
▽ More
Artificial Spiking Neural Networks (ASNNs) promise greater information processing efficiency because of discrete event-based (i.e., spike) computation. Several Machine Learning (ML) applications use biologically inspired plasticity mechanisms as unsupervised learning techniques to increase the robustness of ASNNs while preserving efficiency. Spike Time Dependent Plasticity (STDP) and Intrinsic Plasticity (IP) (i.e., dynamic spiking threshold adaptation) are two such mechanisms that have been combined to form an ensemble learning method. However, it is not clear how this ensemble learning should be regulated based on spiking activity. Moreover, previous studies have attempted threshold based synaptic pruning following STDP, to increase inference efficiency at the cost of performance in ASNNs. However, this type of structural adaptation, that employs individual weight mechanisms, does not consider spiking activity for pruning which is a better representation of input stimuli. We envisaged that plasticity-based spike-regulation and spike-based pruning will result in ASSNs that perform better in low resource situations. In this paper, a novel ensemble learning method based on entropy and network activation is introduced, which is amalgamated with a spike-rate neuron pruning technique, operated exclusively using spiking activity. Two electroencephalography (EEG) datasets are used as the input for classification experiments with a three-layer feed forward ASNN trained using one-pass learning. During the learning process, we observed neurons assembling into a hierarchy of clusters based on spiking rate. It was discovered that pruning lower spike-rate neuron clusters resulted in increased generalization or a predictable decline in performance.
△ Less
Submitted 10 March, 2022;
originally announced March 2022.
