Title:An IR-based Evaluation Framework for Web Search Query Segmentation

Abstract:In this paper, we present a comparative evaluation scheme for Web search query segmentation, based directly on IR performance. We evaluate six segmentation strategies, including four state-of-the-art techniques, vis-a-vis segmentations from three human experts. In the past, segmentation strategies were mainly validated against manual annotations, which suffer from guideline inadequacies and human idiosyncrasy. This work shows that there need not be a perfect correlation between the goodness of a scheme as judged against a handful of human segmentations and its effectiveness from an IR perspective. Moreover, algorithms are shown to perform equally good and sometimes even better in comparison to human markups, a fact masked by previous validations. A test set of relatively rarer queries (more suitable for discriminating between algorithms) were used for evaluation. Our results also prove the usefulness of query segmentation in a Web search scenario.