Predicting query performance using query, result, and user….pptx

Predicting Query Performance Using Query, Result, and User Interaction Features Qi Guo Emory University Ryen White, Susan Dumais, Jue Wang, Blake Anderson Microsoft Presented by Tetsuya Sakai, Microsoft Research 2 Motivation ? Query evaluation is critical for search engines ? Understanding the quality of search results for individual queries (or in the aggregate )? Query evaluation often involves: ? Time-consuming and expensive human judgments ? User studies covering only a small fraction of queries ? Automated methods can lead to rapid and cost-effective query performance prediction ? Prior work used features of queries, results, the collection ? . , Query clarity (Cronen-Townsend et al., 2002); query difficulty prediction (Hauff et al., 2008) 3 Contribution ? Our work differs from previous research: ? Investigates query, results, and interaction features ? Uses search engine logs (rather than standard IR test collections), since they reflect diversity of Web search tasks ? Contributions: ? Investigate novel and rich set of interaction features in predicting query performance ? Determine which features binations of features are more important in predicting query quality ? Understand how accuracy varies with query frequency 4 Predicting Query Performance ? Measured using DCG at rank position 3 (******@3) ? Captures relevance of top-ranked search results ? Relevance of each result measured on five-point scale ? Range of score [0, ] - normalized to [0, 1] ? Goal is to develop a model that can accurately predict ******@3 ? We use DCG (rather than NDCG) because it is an absolute performance score, not normalized by the score of other rankers ? 5 Features ? Three classes: ? Query from Bing search logs ? . , QueryLength, HasURLFragment, HasSpellCorrection ? Results from Bing results pages ? ., AvgNumAds, AvgNumResults, MaxBM25F ? Text-matching baseline ? Interaction from Bing search logs and MSN toolbar logs ? . , AvgClickPos, AvgClickDwe