Skip to main content

Showing 1–16 of 16 results for author: Sarikaya, R

.
  1. Human-AI Interactions Through A Gricean Lens

    Authors: Laura Panfili, Steve Duman, Andrew Nave, Katherine Phelps Ridgeway, Nathan Eversole, Ruhi Sarikaya

    Abstract: Grice's Cooperative Principle (1975) describes the implicit maxims that guide conversation between humans. As humans begin to interact with non-human dialogue systems more frequently and in a broader scope, an important question emerges: what principles govern those interactions? The present study addresses this question by evaluating human-AI interactions using Grice's four maxims; we demonstrate… ▽ More

    Submitted 16 June, 2021; originally announced June 2021.

    Journal ref: Proceedings of the Linguistic Society of America 6 (2021) 288-302

  2. arXiv:2106.02363  [pdf, other

    cs.LG cs.CL

    Learning Slice-Aware Representations with Mixture of Attentions

    Authors: Cheng Wang, Sungjin Lee, Sunghyun Park, Han Li, Young-Bum Kim, Ruhi Sarikaya

    Abstract: Real-world machine learning systems are achieving remarkable performance in terms of coarse-grained metrics like overall accuracy and F-1 score. However, model improvement and development often require fine-grained modeling on individual data subsets or slices, for instance, the data slices where the models have unsatisfactory results. In practice, it gives tangible values for developing such mode… ▽ More

    Submitted 4 June, 2021; originally announced June 2021.

    Comments: Findings of the ACL: ACL-IJCNLP 2021

  3. arXiv:2104.13216  [pdf, other

    cs.LG cs.AI

    Handling Long-Tail Queries with Slice-Aware Conversational Systems

    Authors: Cheng Wang, Sun Kim, Taiwoo Park, Sajal Choudhary, Sunghyun Park, Young-Bum Kim, Ruhi Sarikaya, Sungjin Lee

    Abstract: We have been witnessing the usefulness of conversational AI systems such as Siri and Alexa, directly impacting our daily lives. These systems normally rely on machine learning models evolving over time to provide quality user experience. However, the development and improvement of the models are challenging because they need to support both high (head) and low (tail) usage scenarios, requiring fin… ▽ More

    Submitted 26 April, 2021; originally announced April 2021.

    Comments: Published at ICLR 2021 Workshop on Weakly Supervised Learning

  4. arXiv:2103.03373  [pdf, other

    cs.CL cs.AI cs.LG

    Neural model robustness for skill routing in large-scale conversational AI systems: A design choice exploration

    Authors: Han Li, Sunghyun Park, Aswarth Dara, Jinseok Nam, Sungjin Lee, Young-Bum Kim, Spyros Matsoukas, Ruhi Sarikaya

    Abstract: Current state-of-the-art large-scale conversational AI or intelligent digital assistant systems in industry comprises a set of components such as Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU). For some of these systems that leverage a shared NLU ontology (e.g., a centralized intent/slot schema), there exists a separate skill routing component to correctly route a requ… ▽ More

    Submitted 4 March, 2021; originally announced March 2021.

  5. arXiv:2010.12251  [pdf, other

    cs.CL

    A scalable framework for learning from implicit user feedback to improve natural language understanding in large-scale conversational AI systems

    Authors: Sunghyun Park, Han Li, Ameen Patel, Sidharth Mudgal, Sungjin Lee, Young-Bum Kim, Spyros Matsoukas, Ruhi Sarikaya

    Abstract: Natural Language Understanding (NLU) is an established component within a conversational AI or digital assistant system, and it is responsible for producing semantic understanding of a user request. We propose a scalable and automatic approach for improving NLU in a large-scale conversational AI system by leveraging implicit user feedback, with an insight that user interaction data and dialog cont… ▽ More

    Submitted 10 September, 2021; v1 submitted 23 October, 2020; originally announced October 2020.

    Comments: EMNLP 2021

    ACM Class: I.2.7; I.2.1

  6. arXiv:2006.07113  [pdf, other

    cs.HC cs.AI cs.LG stat.ML

    Large-scale Hybrid Approach for Predicting User Satisfaction with Conversational Agents

    Authors: Dookun Park, Hao Yuan, Dongmin Kim, Yinglei Zhang, Matsoukas Spyros, Young-Bum Kim, Ruhi Sarikaya, Edward Guo, Yuan Ling, Kevin Quinn, Pham Hung, Benjamin Yao, Sungjin Lee

    Abstract: Measuring user satisfaction level is a challenging task, and a critical component in developing large-scale conversational agent systems serving the needs of real users. An widely used approach to tackle this is to collect human annotation data and use them for evaluation or modeling. Human annotation based approaches are easier to control, but hard to scale. A novel alternative approach is to col… ▽ More

    Submitted 29 May, 2020; originally announced June 2020.

  7. arXiv:1911.02557  [pdf, other

    cs.LG cs.AI

    Feedback-Based Self-Learning in Large-Scale Conversational AI Agents

    Authors: Pragaash Ponnusamy, Alireza Roshan Ghias, Chenlei Guo, Ruhi Sarikaya

    Abstract: Today, most large-scale conversational AI agents (e.g. Alexa, Siri, or Google Assistant) are built using manually annotated data to train the different components of the system. Typically, the accuracy of the ML models in these components are improved by manually transcribing and annotating data. As the scope of these systems increase to cover more scenarios and domains, manual annotation to impro… ▽ More

    Submitted 6 November, 2019; originally announced November 2019.

    Comments: 8 pages, 2 figures

  8. arXiv:1905.00924  [pdf, other

    cs.LG cs.CL stat.ML

    Locale-agnostic Universal Domain Classification Model in Spoken Language Understanding

    Authors: Jihwan Lee, Ruhi Sarikaya, Young-Bum Kim

    Abstract: In this paper, we introduce an approach for leveraging available data across multiple locales sharing the same language to 1) improve domain classification model accuracy in Spoken Language Understanding and user experience even if new locales do not have sufficient data and 2) reduce the cost of scaling the domain classifier to a large number of locales. We propose a locale-agnostic universal dom… ▽ More

    Submitted 2 May, 2019; originally announced May 2019.

    Comments: NAACL-HLT 2019

  9. arXiv:1905.00921  [pdf, other

    cs.LG cs.AI stat.ML

    Continuous Learning for Large-scale Personalized Domain Classification

    Authors: Han Li, Jihwan Lee, Sidharth Mudgal, Ruhi Sarikaya, Young-Bum Kim

    Abstract: Domain classification is the task of mapping spoken language utterances to one of the natural language understanding domains in intelligent personal digital assistants (IPDAs). This is a major component in mainstream IPDAs in industry. Apart from official domains, thousands of third-party domains are also created by external developers to enhance the capability of IPDAs. As more domains are develo… ▽ More

    Submitted 2 May, 2019; originally announced May 2019.

    Comments: NAACL-HLT 2019

  10. arXiv:1812.06083  [pdf, other

    cs.CL cs.LG stat.ML

    Coupled Representation Learning for Domains, Intents and Slots in Spoken Language Understanding

    Authors: JIhwan Lee, Dongchan Kim, Ruhi Sarikaya, Young-Bum Kim

    Abstract: Representation learning is an essential problem in a wide range of applications and it is important for performing downstream tasks successfully. In this paper, we propose a new model that learns coupled representations of domains, intents, and slots by taking advantage of their hierarchical dependency in a Spoken Language Understanding system. Our proposed model learns the vector representation o… ▽ More

    Submitted 13 December, 2018; originally announced December 2018.

    Comments: IEEE SLT 2018

  11. arXiv:1810.12464  [pdf, other

    cs.LG stat.ML

    Differentiable Greedy Networks

    Authors: Thomas Powers, Rasool Fakoor, Siamak Shakeri, Abhinav Sethy, Amanjit Kainth, Abdel-rahman Mohamed, Ruhi Sarikaya

    Abstract: Optimal selection of a subset of items from a given set is a hard problem that requires combinatorial optimization. In this paper, we propose a subset selection algorithm that is trainable with gradient-based methods yet achieves near-optimal performance via submodular optimization. We focus on the task of identifying a relevant set of sentences for claim verification in the context of the FEVER t… ▽ More

    Submitted 29 October, 2018; originally announced October 2018.

    Comments: Work in progress and under review

  12. arXiv:1810.00679  [pdf, other

    cs.IR cs.CL cs.LG stat.ML

    Direct optimization of F-measure for retrieval-based personal question answering

    Authors: Rasool Fakoor, Amanjit Kainth, Siamak Shakeri, Christopher Winestock, Abdel-rahman Mohamed, Ruhi Sarikaya

    Abstract: Recent advances in spoken language technologies and the introduction of many customer facing products, have given rise to a wide customer reliance on smart personal assistants for many of their daily tasks. In this paper, we present a system to reduce users' cognitive load by extending personal assistants with long-term personal memory where users can store and retrieve by voice, arbitrary pieces… ▽ More

    Submitted 27 September, 2018; originally announced October 2018.

    Comments: accepted at SLT2018

  13. Contextual Slot Carryover for Disparate Schemas

    Authors: Chetan Naik, Arpit Gupta, Hancheng Ge, Lambert Mathias, Ruhi Sarikaya

    Abstract: In the slot-filling paradigm, where a user can refer back to slots in the context during a conversation, the goal of the contextual understanding system is to resolve the referring expressions to the appropriate slots in the context. In large-scale multi-domain systems, this presents two challenges - scaling to a very large and potentially unbounded set of slot values, and dealing with diverse sch… ▽ More

    Submitted 5 June, 2018; originally announced June 2018.

    Comments: Accepted at Interspeech 2018

  14. arXiv:1804.08065  [pdf, other

    cs.CL

    Efficient Large-Scale Domain Classification with Personalized Attention

    Authors: Young-Bum Kim, Dongchan Kim, Anjishnu Kumar, Ruhi Sarikaya

    Abstract: In this paper, we explore the task of mapping spoken language utterances to one of thousands of natural language understanding domains in intelligent personal digital assistants (IPDAs). This scenario is observed for many mainstream IPDAs in industry that allow third parties to develop thousands of new domains to augment built-in ones to rapidly increase domain coverage and overall IPDA capabiliti… ▽ More

    Submitted 22 April, 2018; originally announced April 2018.

    Comments: Accepted to ACL 2018

  15. arXiv:1804.08064  [pdf, other

    cs.CL

    A Scalable Neural Shortlisting-Reranking Approach for Large-Scale Domain Classification in Natural Language Understanding

    Authors: Young-Bum Kim, Dongchan Kim, Joo-Kyung Kim, Ruhi Sarikaya

    Abstract: Intelligent personal digital assistants (IPDAs), a popular real-life application with spoken language understanding capabilities, can cover potentially thousands of overlapping domains for natural language understanding, and the task of finding the best domain to handle an utterance becomes a challenging problem on a large scale. In this paper, we propose a set of efficient and scalable neural sho… ▽ More

    Submitted 21 April, 2018; originally announced April 2018.

    Comments: Accepted to NAACL 2018

  16. arXiv:1711.10705  [pdf, other

    cs.CL

    Speaker-Sensitive Dual Memory Networks for Multi-Turn Slot Tagging

    Authors: Young-Bum Kim, Sungjin Lee, Ruhi Sarikaya

    Abstract: In multi-turn dialogs, natural language understanding models can introduce obvious errors by being blind to contextual information. To incorporate dialog history, we present a neural architecture with Speaker-Sensitive Dual Memory Networks which encode utterances differently depending on the speaker. This addresses the different extents of information available to the system - the system knows onl… ▽ More

    Submitted 29 November, 2017; originally announced November 2017.

    Comments: 5 pages conference paper accepted to IEEE ASRU 2017. Will be published in December 2017