A Methodology for Extrinsic Evaluation of Text Summarization: Does ROUGE Correlate? (2005)
Bonnie Dorr, Christof Monz, Stacy President, Richard Schwartz, David Zajic
This paper demonstrates the usefulness of summaries in an extrinsic task of relevance judgment based on a new method for measuring agreement, Relevance-Prediction, which compares subjects '...
Extrinsic Evaluation of Automatic Metrics for Summarization (2004)
Bonnie Dorr, Christof Monz, Douglas Oard, Stacy President, David Zajic, Richard Schwartz
This paper describes extrinsic-task evaluation of summarization. We show that it is possible to save time using summaries for relevance assessment without adversely impacting the degree of accuracy...
Extrinsic Evaluation of Automatic Metrics for Summarization (1998)
Dorr, Bonnie, Monz, Christof, Oard, Douglas, President, Stacy, Zajic, David, Schwartz, Richard
This paper describes extrinsic-task evaluation of summarization. We show that it is possible to save time using summaries for relevance assessment without adversely impacting the degree of accuracy...