Title:Topic Augmented Neural Network for Short Text Conversation

Abstract:We consider matching input messages with proper responses for short-text conversation. The matching should be performed not only by the messages and the responses but also by the topics of the messages. To this end, we propose a topic augmented neural network which consists of a sentence embedding layer, a topic embedding layer, and a matching layer. The sentence embedding layer embeds an input message and a response into a vector space, while the topic embedding layer forms a topic vector by a linear combination of the embedding of the topic words whose weights are determined by both themselves and the message vector. The message vector, the response vector, and the topic vector are then fed to the matching layer to calculate the matching score between the message and the response. Empirical study on large scale annotated data shows that our model can significantly outperform state of the art matching models.