| 1998 Broadcast News Benchmark Test Results: English and Non-English Word Error Rate Performance Measures (1999) | |||||||||||||||
Abstract | |||||||||||||||
| This paper documents the use of Broadcast News test materials in DARPA-sponsored Automatic Speech Recognition (ASR) Benchmark Tests conducted late in 1998. As in last year’s tests [1], statistical selection procedures were used in selecting test materials. Two test epochs were used, each yielding (nominally) one and one-half hours of test material. One of the test sets was drawn from the same test epoch as was used for last year’s tests, and the other was drawn from a more recent period. Results are reported for two types of systems: one (the “Hub”, or “baseline ” systems) for which there were no limits on computational resources, and another (the “less than 10X realtime spoke ” systems) for systems that ran in less than 10 times real-time. The lowest word error rate reported this year for the “Hub” systems was 13.5%, contrasting with last year’s lowest word error rate of 16.2%. For the “less than 10X real-time spoke” systems, the lowest reported word error rate was 16.1%. Results are also reported, for the second year, on non-English language Broadcast News materials in Spanish and Mandarin. 1. TEST MATERIALS | |||||||||||||||
Publication details | |||||||||||||||
| |||||||||||||||