| Overview of the trec-9 question answering track (2000) | |||||||||||||||
Abstract | |||||||||||||||
| The TREC question answering track is an eort to bring the benets of large-scale evaluation to bear on the question answering problem. The track has run twice so far, where the goal both times was to retrieve small snippets of text that contain the actual answer to a question rather than the document lists traditionally returned by text retrieval systems. The best performing system in TREC-9, the Falcon system from Southern Methodist University, was able to answer about 65 % of the questions (compared to approximately 42 % of the questions for the next best systems) by combining abductive reasoning with various natural language processing techniques. The 65 % score is slightly less than the best scores for TREC-8 in absolute terms, but it represents a very signicant improvement in question answering systems. The TREC-9 task was considerably harder than the TREC-8 task because the TREC-9 track used actual users ' questions rather than questions constructed specically for the track. The TREC-9 question answering (QA) track was the second running of a QA track in TREC. The goal of the track is to foster research on systems that retrieve answers rather than documents in response to a question, with an emphasis on systems that can function in unrestricted domains. While the subject matter of the questions is not restricted, the type of questions the systems are expected to process is limited. | |||||||||||||||
Publication details | |||||||||||||||
| |||||||||||||||