| Overview of the TREC 2002 question answering track (2002) | |||||||||||||||
Abstract | |||||||||||||||
| The TREC question answering track is an effort to bring the benefits of loxge-scMe evauation to beox on the question nswering problem. The track contained two tasks in TREC 2002, the min task nd the list task. Both tasks required that the nswer strings returned by the systems consist of nothing more or less thn n answer in contrast to the text snippets containing n answer alowed in previous yeoxs. A new evaluation measure in the min task, the confidence-weighted score, tested a system's ability to recognize when it has found a correct nswer. The goal of the question answering (QA) track is to foster research on systems that retrieve answers rather than documents in response to a question, with particular emphasis on systems that can function in unrestricted domains. Now in its fourth year, the tasks in the track have evolved over the years to increase the realism of the task and to focus research on particular aspects of the problem deemed important to improving the state-of-the-art. All of the tasks have involved finding answers to closed-class questions within a large corpus of news text. This paper provides an overview of the TREC 2002 QA track. This year's track contained two tasks, the main task and the list task. Both tasks were also run in TREC 2001, but systems were required to return exact answers this year. That is, the text string returned by the system in response to a question was required to consist of a complete answer and nothing else, in contrast to earlier years where systems could return text strings that simply contained an answer. To make the paper self-contained, the first section recaps the tasks and evaluation procedures used in the first three tracks. The following sections then describe this year's tasks. | |||||||||||||||
Publication details | |||||||||||||||
| |||||||||||||||