Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering
Authors:
Zhentao Xu,
Mark Jerome Cruz,
Matthew Guevara,
Tie Wang,
Manasi Deshpande,
Xiaofeng Wang,
Zheng Li
Abstract:
In customer service technical support, swiftly and accurately retrieving relevant past issues is critical for efficiently resolving customer inquiries. The conventional retrieval methods in retrieval-augmented generation (RAG) for large language models (LLMs) treat a large corpus of past issue tracking tickets as plain text, ignoring the crucial intra-issue structure and inter-issue relations, whi…
▽ More
In customer service technical support, swiftly and accurately retrieving relevant past issues is critical for efficiently resolving customer inquiries. The conventional retrieval methods in retrieval-augmented generation (RAG) for large language models (LLMs) treat a large corpus of past issue tracking tickets as plain text, ignoring the crucial intra-issue structure and inter-issue relations, which limits performance. We introduce a novel customer service question-answering method that amalgamates RAG with a knowledge graph (KG). Our method constructs a KG from historical issues for use in retrieval, retaining the intra-issue structure and inter-issue relations. During the question-answering phase, our method parses consumer queries and retrieves related sub-graphs from the KG to generate answers. This integration of a KG not only improves retrieval accuracy by preserving customer service structure information but also enhances answering quality by mitigating the effects of text segmentation. Empirical assessments on our benchmark datasets, utilizing key retrieval (MRR, Recall@K, NDCG@K) and text generation (BLEU, ROUGE, METEOR) metrics, reveal that our method outperforms the baseline by 77.6% in MRR and by 0.32 in BLEU. Our method has been deployed within LinkedIn's customer service team for approximately six months and has reduced the median per-issue resolution time by 28.6%.
△ Less
Submitted 6 May, 2024; v1 submitted 26 April, 2024;
originally announced April 2024.
Level of Awareness of PSU Bayambang Campus Students towards E learning Technologies
Authors:
Matthew John F. Sino Cruz,
Kim Eric B. Nanlabi,
Michael Ryan C. Peoro
Abstract:
The study assesses the awareness of PSU Bayambang Campus students regarding e-learning technologies. A Quantitative Research Approach was used, gathering data through a demographic questionnaire and ICT Resources assessment. The survey measured students' familiarity and knowledge of existing e-learning technologies. Around 52.50% of respondents were familiar with e learning concepts, but their exp…
▽ More
The study assesses the awareness of PSU Bayambang Campus students regarding e-learning technologies. A Quantitative Research Approach was used, gathering data through a demographic questionnaire and ICT Resources assessment. The survey measured students' familiarity and knowledge of existing e-learning technologies. Around 52.50% of respondents were familiar with e learning concepts, but their exposure and utilization levels need consideration. Technology, Support, and Users were identified as key factors influencing student awareness. Implementation can be improved through policies and resource provision. The researchers recommend integrating e learning policies, providing ICT Resources and Infrastructure, and offering training for students and teachers. This research serves as a guide for policy design, enhancing the University's learning process and facilitating better learning and interaction.
△ Less
Submitted 6 August, 2023;
originally announced August 2023.
