-
A Review of LLM-Assisted Ideation
Authors:
Sitong Li,
Stefano Padilla,
Pierre Le Bras,
Junyu Dong,
Mike Chantler
Abstract:
We present a comprehensive, in-depth review of ideation assisted by large language models (LLMs), highlighting emerging trends and identifying unaddressed research gaps. In total, we examined 61 studies investigating the application of LLMs in both group and individual ideation processes. From these studies, we derived the Hourglass Ideation Framework for LLM-assisted ideation, comprising three ph…
▽ More
We present a comprehensive, in-depth review of ideation assisted by large language models (LLMs), highlighting emerging trends and identifying unaddressed research gaps. In total, we examined 61 studies investigating the application of LLMs in both group and individual ideation processes. From these studies, we derived the Hourglass Ideation Framework for LLM-assisted ideation, comprising three phases and seven key ideation stages, which served as the basis for our systematic survey. Our analysis reveals that LLMs are most frequently used for idea generation and refinement, but their use in scope specification, foundational material structuring and multi-idea evaluation and selection remains limited. We provide our findings in extensive tabular and online formats. These catalogues detail research on LLM-assisted, purely LLM-based, and human-only activities across the seven ideation stages for each of the 61 studies. These also detail creative domains, publication outlets, interaction designs, user study designs, and assessment methods. Our analysis of system interaction design reveals a predominant focus on supporting individual ideation activities and text-based interaction, with a growing trend of incorporating multimedia elements. However, in group ideation, tools and interaction modalities targeting both synchronous and asynchronous collaboration are much scarcer. We synthesize the primary findings of our review and outline promising directions for future research in LLM-assisted ideation. We hope this review will help researchers quickly gain an overview of this rapidly expanding area, efficiently locate relevant work, and identify underexplored areas for further investigation. In addition, we believe the framework we present here will form the basis for the development of future problem and solution space taxonomies, and methodologies for LLM-assisted ideation development and use.
△ Less
Submitted 4 March, 2025; v1 submitted 2 March, 2025;
originally announced March 2025.
-
Visual Exploration of Stopword Probabilities in Topic Models
Authors:
Shuangjiang Xue,
Pierre Le Bras,
David A. Robb,
Mike J. Chantler,
Stefano Padilla
Abstract:
Stopword removal is a critical stage in many Machine Learning methods but often receives little consideration, it interferes with the model visualizations and disrupts user confidence. Inappropriately chosen or hastily omitted stopwords not only lead to suboptimal performance but also significantly affect the quality of models, thus reducing the willingness of practitioners and stakeholders to rel…
▽ More
Stopword removal is a critical stage in many Machine Learning methods but often receives little consideration, it interferes with the model visualizations and disrupts user confidence. Inappropriately chosen or hastily omitted stopwords not only lead to suboptimal performance but also significantly affect the quality of models, thus reducing the willingness of practitioners and stakeholders to rely on the output visualizations. This paper proposes a novel extraction method that provides a corpus-specific probabilistic estimation of stopword likelihood and an interactive visualization system to support their analysis. We evaluated our approach and interface using real-world data, a commonly used Machine Learning method (Topic Modelling), and a comprehensive qualitative experiment probing user confidence. The results of our work show that our system increases user confidence in the credibility of topic models by (1) returning reasonable probabilities, (2) generating an appropriate and representative extension of common stopword lists, and (3) providing an adjustable threshold for estimating and analyzing stopwords visually. Finally, we discuss insights, recommendations, and best practices to support practitioners while improving the output of Machine Learning methods and topic model visualizations with robust stopword analysis and removal.
△ Less
Submitted 17 January, 2025;
originally announced January 2025.
-
Three Approaches to the Automation of Laser System Alignment and Their Resource Implications: A Case Study
Authors:
David A. Robb,
Donald Risbridger,
Ben Mills,
Ildar Rakhmatulin,
Xianwen Kong,
Mustafa Erden,
M. J. Daniel Esser,
Richard M. Carter,
Mike J. Chantler
Abstract:
The alignment of optical systems is a critical step in their manufacture. Alignment normally requires considerable knowledge and expertise of skilled operators. The automation of such processes has several potential advantages, but requires additional resource and upfront costs. Through a case study of a simple two mirror system we identify and examine three different automation approaches. They a…
▽ More
The alignment of optical systems is a critical step in their manufacture. Alignment normally requires considerable knowledge and expertise of skilled operators. The automation of such processes has several potential advantages, but requires additional resource and upfront costs. Through a case study of a simple two mirror system we identify and examine three different automation approaches. They are: artificial neural networks; practice-led, which mimics manual alignment practices; and design-led, modelling from first principles. We find that these approaches make use of three different types of knowledge 1) basic system knowledge (of controls, measurements and goals); 2) behavioural skills and expertise, and 3) fundamental system design knowledge. We demonstrate that the different automation approaches vary significantly in human resources, and measurement sampling budgets. This will have implications for practitioners and management considering the automation of such tasks.
△ Less
Submitted 17 September, 2024;
originally announced September 2024.
-
Enhancing Reading Strategies by Exploring A Theme-based Approach to Literature Surveys
Authors:
Tanya Howden,
Pierre Le Bras,
Thomas S. Methven,
Stefano Padilla,
Mike J. Chantler
Abstract:
Searching large digital repositories can be extremely frustrating, as common list-based formats encourage users to adopt a convenience-sampling approach that favours chance discovery and random search, over meaningful exploration. We have designed a methodology that allows users to visually and thematically explore corpora, while developing personalised holistic reading strategies. We describe the…
▽ More
Searching large digital repositories can be extremely frustrating, as common list-based formats encourage users to adopt a convenience-sampling approach that favours chance discovery and random search, over meaningful exploration. We have designed a methodology that allows users to visually and thematically explore corpora, while developing personalised holistic reading strategies. We describe the results of a three-phase qualitative study, in which experienced researchers used our interactive visualisation approach to analyse a set of publications and select relevant themes and papers. Using in-depth semi-structured interviews and stimulated recall, we found that users: (i) selected papers that they otherwise would not have read, (ii) developed a more coherent reading strategy, and (iii) understood the thematic structure and relationships between papers more effectively. Finally, we make six design recommendations to enhance current digital repositories that we have shown encourage users to adopt a more holistic and thematic research approach.
△ Less
Submitted 10 February, 2021;
originally announced February 2021.
-
Visualising COVID-19 Research
Authors:
Pierre Le Bras,
Azimeh Gharavi,
David A. Robb,
Ana F. Vidal,
Stefano Padilla,
Mike J. Chantler
Abstract:
The world has seen in 2020 an unprecedented global outbreak of SARS-CoV-2, a new strain of coronavirus, causing the COVID-19 pandemic, and radically changing our lives and work conditions. Many scientists are working tirelessly to find a treatment and a possible vaccine. Furthermore, governments, scientific institutions and companies are acting quickly to make resources available, including funds…
▽ More
The world has seen in 2020 an unprecedented global outbreak of SARS-CoV-2, a new strain of coronavirus, causing the COVID-19 pandemic, and radically changing our lives and work conditions. Many scientists are working tirelessly to find a treatment and a possible vaccine. Furthermore, governments, scientific institutions and companies are acting quickly to make resources available, including funds and the opening of large-volume data repositories, to accelerate innovation and discovery aimed at solving this pandemic. In this paper, we develop a novel automated theme-based visualisation method, combining advanced data modelling of large corpora, information mapping and trend analysis, to provide a top-down and bottom-up browsing and search interface for quick discovery of topics and research resources. We apply this method on two recently released publications datasets (Dimensions' COVID-19 dataset and the Allen Institute for AI's CORD-19). The results reveal intriguing information including increased efforts in topics such as social distancing; cross-domain initiatives (e.g. mental health and education); evolving research in medical topics; and the unfolding trajectory of the virus in different territories through publications. The results also demonstrate the need to quickly and automatically enable search and browsing of large corpora. We believe our methodology will improve future large volume visualisation and discovery systems but also hope our visualisation interfaces will currently aid scientists, researchers, and the general public to tackle the numerous issues in the fight against the COVID-19 pandemic.
△ Less
Submitted 15 May, 2020; v1 submitted 13 May, 2020;
originally announced May 2020.
-
Challenges in Collaborative HRI for Remote Robot Teams
Authors:
Helen Hastie,
David A. Robb,
José Lopes,
Muneeb Ahmad,
Pierre Le Bras,
Xingkun Liu,
Ronald P. A. Petrick,
Katrin Lohan,
Mike J. Chantler
Abstract:
Collaboration between human supervisors and remote teams of robots is highly challenging, particularly in high-stakes, distant, hazardous locations, such as off-shore energy platforms. In order for these teams of robots to truly be beneficial, they need to be trusted to operate autonomously, performing tasks such as inspection and emergency response, thus reducing the number of personnel placed in…
▽ More
Collaboration between human supervisors and remote teams of robots is highly challenging, particularly in high-stakes, distant, hazardous locations, such as off-shore energy platforms. In order for these teams of robots to truly be beneficial, they need to be trusted to operate autonomously, performing tasks such as inspection and emergency response, thus reducing the number of personnel placed in harm's way. As remote robots are generally trusted less than robots in close-proximity, we present a solution to instil trust in the operator through a `mediator robot' that can exhibit social skills, alongside sophisticated visualisation techniques. In this position paper, we present general challenges and then take a closer look at one challenge in particular, discussing an initial study, which investigates the relationship between the level of control the supervisor hands over to the mediator robot and how this affects their trust. We show that the supervisor is more likely to have higher trust overall if their initial experience involves handing over control of the emergency situation to the robotic assistant. We discuss this result, here, as well as other challenges and interaction techniques for human-robot collaboration.
△ Less
Submitted 17 May, 2019;
originally announced May 2019.
-
The ORCA Hub: Explainable Offshore Robotics through Intelligent Interfaces
Authors:
Helen Hastie,
Katrin Lohan,
Mike Chantler,
David A. Robb,
Subramanian Ramamoorthy,
Ron Petrick,
Sethu Vijayakumar,
David Lane
Abstract:
We present the UK Robotics and Artificial Intelligence Hub for Offshore Robotics for Certification of Assets (ORCA Hub), a 3.5 year EPSRC funded, multi-site project. The ORCA Hub vision is to use teams of robots and autonomous intelligent systems (AIS) to work on offshore energy platforms to enable cheaper, safer and more efficient working practices. The ORCA Hub will research, integrate, validate…
▽ More
We present the UK Robotics and Artificial Intelligence Hub for Offshore Robotics for Certification of Assets (ORCA Hub), a 3.5 year EPSRC funded, multi-site project. The ORCA Hub vision is to use teams of robots and autonomous intelligent systems (AIS) to work on offshore energy platforms to enable cheaper, safer and more efficient working practices. The ORCA Hub will research, integrate, validate and deploy remote AIS solutions that can operate with existing and future offshore energy assets and sensors, interacting safely in autonomous or semi-autonomous modes in complex and cluttered environments, co-operating with remote operators. The goal is that through the use of such robotic systems offshore, the need for personnel will decrease. To enable this to happen, the remote operator will need a high level of situation awareness and key to this is the transparency of what the autonomous systems are doing and why. This increased transparency will facilitate a trusting relationship, which is particularly key in high-stakes, hazardous situations.
△ Less
Submitted 6 March, 2018;
originally announced March 2018.
-
A 'Gibbs-Newton' Technique for Enhanced Inference of Multivariate Polya Parameters and Topic Models
Authors:
Osama Khalifa,
David Wolfe Corne,
Mike Chantler
Abstract:
Hyper-parameters play a major role in the learning and inference process of latent Dirichlet allocation (LDA). In order to begin the LDA latent variables learning process, these hyper-parameters values need to be pre-determined. We propose an extension for LDA that we call 'Latent Dirichlet allocation Gibbs Newton' (LDA-GN), which places non-informative priors over these hyper-parameters and uses…
▽ More
Hyper-parameters play a major role in the learning and inference process of latent Dirichlet allocation (LDA). In order to begin the LDA latent variables learning process, these hyper-parameters values need to be pre-determined. We propose an extension for LDA that we call 'Latent Dirichlet allocation Gibbs Newton' (LDA-GN), which places non-informative priors over these hyper-parameters and uses Gibbs sampling to learn appropriate values for them. At the heart of LDA-GN is our proposed 'Gibbs-Newton' algorithm, which is a new technique for learning the parameters of multivariate Polya distributions. We report Gibbs-Newton performance results compared with two prominent existing approaches to the latter task: Minka's fixed-point iteration method and the Moments method. We then evaluate LDA-GN in two ways: (i) by comparing it with standard LDA in terms of the ability of the resulting topic models to generalize to unseen documents; (ii) by comparing it with standard LDA in its performance on a binary classification task.
△ Less
Submitted 27 February, 2016; v1 submitted 22 October, 2015;
originally announced October 2015.
