Showing 1–2 of 2 results for author: Jiang, J J

Look Ahead Text Understanding and LLM Stitching

Authors: Junlin Julian Jiang, Xin Li

Abstract: This paper proposes a look ahead text understanding problem with look ahead section identification (LASI) as an example. This problem may appear in generative AI as well as human interactions, where we want to understand the direction of a developing text or conversation. We tackle the problem using transformer-based LLMs. We show that LASI is more challenging than classic section identification (… ▽ More This paper proposes a look ahead text understanding problem with look ahead section identification (LASI) as an example. This problem may appear in generative AI as well as human interactions, where we want to understand the direction of a developing text or conversation. We tackle the problem using transformer-based LLMs. We show that LASI is more challenging than classic section identification (SI). We argue that both bidirectional contextual information (e.g., BERT) and unidirectional predictive ability (e.g., GPT) will benefit the task. We propose two approaches to stitch together BERT and GPT. Experiments show that our approach outperforms the established models, especially when there is noise in the text (which is often the case for developing text in generative AI). Our paper sheds light on other look ahead text understanding tasks that are important to social media, such as look ahead sentiment classification, and points out the opportunities to leverage pre-trained LLMs through stitching. △ Less

Submitted 15 December, 2024; originally announced December 2024.

Comments: 9 pages, 6 figures, 4 tables, published in Vol. 18 (2024): Proceedings of the Eighteenth International AAAI Conference on Web and Social Media

Journal ref: Proceedings of the International AAAI Conference on Web and Social Media, 18(1), 751-760 (2024)

Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy

Authors: Jay J. Jiang, David W. Conrath

Abstract: This paper presents a new approach for measuring semantic similarity/distance between words and concepts. It combines a lexical taxonomy structure with corpus statistical information so that the semantic distance between nodes in the semantic space constructed by the taxonomy can be better quantified with the computational evidence derived from a distributional analysis of corpus data. Specifica… ▽ More This paper presents a new approach for measuring semantic similarity/distance between words and concepts. It combines a lexical taxonomy structure with corpus statistical information so that the semantic distance between nodes in the semantic space constructed by the taxonomy can be better quantified with the computational evidence derived from a distributional analysis of corpus data. Specifically, the proposed measure is a combined approach that inherits the edge-based approach of the edge counting scheme, which is then enhanced by the node-based approach of the information content calculation. When tested on a common data set of word pair similarity ratings, the proposed approach outperforms other computational models. It gives the highest correlation value (r = 0.828) with a benchmark based on human similarity judgements, whereas an upper bound (r = 0.885) is observed when human subjects replicate the same task. △ Less

Submitted 20 September, 1997; originally announced September 1997.

Comments: 15 pages, Postscript only

Journal ref: In the Proceedings of ROCLING X, Taiwan, 1997