Minimal Feedback during Tutorial Dialogue ⋆ (2009)
Abstract. We analyzed unexpected student responses to a natural language (NL) ITS to determine if improved feedback could be beneficial. Our analysis of a corpus of ITS-student dialogues suggests...
Content-Learning Correlations in Spoken Tutoring Dialogs at Word, Turn and Discourse Levels (2009)
We study correlations between dialog content and learning in a corpus of human-computer tutoring dialogs. Using an online encyclopedia, we first extract domainspecific concepts discussed in our...
Minimal Feedback during Tutorial Dialogue ⋆ (2009)
Abstract. We analyzed unexpected student responses to a natural language (NL) ITS to determine if improved feedback could be beneficial. Our analysis of a corpus of ITS-student dialogues suggests...
Comparing Spoken Dialog Corpora Collected with Recruited Subjects versus Real Users (2009)
Hua Ai, Antoine Raux, Dan Bohus, Maxine Eskenazi, Diane Litman
Empirical spoken dialog research often involves the collection and analysis of a dialog corpus. However, it is not well understood whether and how a corpus of dialogs collected using recruited...
2008a. Responding to student uncertainty during computer tutoring: A preliminary evaluation (2009)
Kate Forbes-riley, Diane Litman, Mihai Rotaru
Abstract. This paper evaluates dialogue-based student performance in a controlled experiment using versions of a tutoring system with and without automatic adaptation to the student affective state...
Semantic Cohesion and Learning (2009)
Abstract. A previously reported measure of dialog cohesion was extended to measure cohesion by counting semantic similarity (the repetition of meaning) as well as lexical reiteration (the repetition...
Comparing Spoken Dialog Corpora Collected with Recruited Subjects versus Real Users (2008)
Hua Ai, Antoine Raux, Dan Bohus, Maxine Eskenazi, Diane Litman
Empirical spoken dialog research often involves the collection and analysis of a dialog corpus. However, it is not well understood whether and how a corpus of dialogs collected using recruited...
Content-Learning Correlations in Spoken Tutoring Dialogs at Word, Turn and Discourse Levels (2008)
We study correlations between dialog content and learning in a corpus of human-computer tutoring dialogs. Using an online encyclopedia, we first extract domainspecific concepts discussed in our...
Janyce Wiebe, Eric Breck, Chris Buckley, Claire Cardie, Paul Davis, Bruce Fraser, ...
Tomorrow’s question answering systems will need to have the ability to process information about beliefs, opinions, and evaluations—the perspective of an agent. Answers to many simple factual
Uncertainty Corpus: Resource to Study User Affect in Complex Spoken Dialogue Systems (2008)
Kate Forbes-riley, Diane Litman, Scott Silliman, Amruta Pur
We present a corpus of spoken dialogues between students and an adaptive Wizard-of-Oz tutoring system, in which student uncertainty was manually annotated in real-time. We detail the corpus contents,...
on Discourse Structure in Natural Language Understanding and Generation, ed. by (2008)
Julia Hirschberg, Diane Litman, Kathy Mccoy, Y Sidner, Pacific Grove
YAROWSKY, DAVID. 1992. Word sense disambiguation using statistical models of Roget’s categories trained on large corpora. In Proceedings of the Fourteenth International Conference on Computational...
Investigating Human Tutor Responses to Student Uncertainty for Adaptive System Development (2008)
Kate Forbes-riley, Diane Litman
Abstract. We use a χ 2 analysis on our spoken dialogue tutoring corpus to investigate dependencies between uncertain student answers and 9 dialogue acts the human tutor uses in his response to these...
Michael Kearns, Diane Litman, Marilyn Walker
Recently, a number of authors have proposed treating dialogue systems as Markov decision processes (MDPs). However, the practical application of MDP algorithms to dialogue systems faces a number of...
D.: Developing pedagogically effective tutorial dialogue tactics: Experiments and a testbed (2008)
Kurt Vanlehn, Pamela Jordan, Diane Litman
Although effective tutorial dialogue strategies are well understood, tutorial tactics that govern brief episodes of tutoring, such as a single step, are not. Because better tactics seem to be crucial...
Manual Annotation of Opinion Categories in Meetings (2008)
Swapna Somasundaran, Janyce Wiebe, Paul Hoffmann, Diane Litman
This paper applies the categories from an opinion annotation scheme developed for monologue text to the genre of multiparty meetings. We describe modifications to the coding guidelines that were...
Veselin Stoyanov, Claire Cardie, Diane Litman, Janyce Wiebe
In recent work, Wiebe et al. (2003) propose a semantic representation for encoding the opinions and perspectives expressed at any given point in a text. This paper evaluates the opinion annotation...
Claire Cardie, Janyce Wiebe, Theresa Wilson, Diane Litman
Abstract 1 While much recent progress has been made in research on fact-based question answering, our work aims to extend question-answering research in a different direction ─ to handle...
R++: Adding Path-Based Rules to C (2007)
Diane Litman, Peter Patel-schneider, Anil Mishra, James Crawford, Daniel Dvorak
Object-oriented languages and rule-based languages offer two distinct and useful programming abstractions. However, previous attempts to integrate data-driven rules into object-oriented languages...
Evaluating Spoken Language Systems (2007)
Candace Kamm And, Ace Kamm, Marilyn Walker, Diane Litman
Spoken language systems (SLSs) for accessing information sources or services through the telephone network and the Internet are currently being trialed and deployed for a variety of tasks. Evaluating...
Evaluating Spoken Language Systems (2007)
Ace Kamm, Marilyn Walker, Diane Litman
Spoken language systems (SLSs) for accessing information sources or services through the telephone network and the Internet are currently being trialed and deployed for a variety of tasks. Evaluating...
Dialog convergence and learning (2007)
Abstract. In this paper we examine whether the student-to-tutor convergence of lexical and speech features is a useful predictor of learning in a corpus of spoken tutorial dialogs. This possibility...
Measuring convergence and priming in tutorial dialog (2007)
Experimental research has shown that human users will converge with dialog systems along many dimensions of speech, including those of acoustic/prosodic features and lexical choice. Other results...
Comparing linguistic features for modeling learning in computer dialogue tutoring (2007)
Kate Forbes-riley, Diane Litman, Amruta Purandare, Mihai Rotaru, Joel Tetreault
Abstract. We compare the relative utility of different automatically computable
Predicting Learning in Tutoring with the Landscape Model of Memory (2006)
A Landscape Model analysis, adopted from the text processing literature, was run on transcripts of tutoring sessions, and a technique developed to count the occurrence of key physics points in the...
Characterizing and predicting corrections in spoken dialogue systems (2006)
Diane Litman, Marc Swerts, Julia Hirschberg
This article focuses on the analysis and prediction of corrections, defined as turns where a user tries to correct a prior error made by a spoken dialogue system. We describe our labeling procedure...
Kate Forbes-riley, Diane Litman, Scott Silliman, Joel Tetreault
We evaluate the impact of tutor voice quality in the context of our intelligent tutoring spoken dialogue system. We first describe two versions of our system which yielded two corpora of...
Comparing real-real, simulated-simulated, and simulated-real spoken dialogue corpora (2006)
User simulation is used to generate large corpora for using reinforcement learning to automatically learn the best policy for spoken dialogue systems. Although this approach is becoming increasingly...
Kate Forbes-riley, Diane Litman
We examine correlations between student learning and student acoustic-prosodic profiles, which prior research has shown to be predictive of emotional states. We compare these correlations in two...
Speech recognition performance and learning in spoken dialogue tutoring (2005)
Diane Litman, Kate Forbes-riley
Speech recognition errors have been shown to negatively correlate with user satisfaction in evaluations of task-oriented spoken dialogue systems. In the domain of tutorial dialogue systems, however,...
Dialogue-learning correlations in spoken dialogue tutoring (2005)
Kate Forbes-riley, Diane Litman, Alison Huettner, Arthur Ward
Abstract. We examine correlations between dialogue characteristics and learning in two corpora of spoken tutoring dialogues: a human-human corpus and a humancomputer corpus, both of which have been...
The goal of my current research is to generate an empirically-based understanding of the ramifications of adding spoken language capabilities and affective computing to text-based dialogue tutors,...
Co-training for Predicting Emotions with Spoken Dialogue Data (2004)
Beatriz Maeireizo And, Beatriz Maeireizo, Diane Litman, Rebecca Hwa
Natural Language Processing applications often require large amounts of annotated training data, which are expensive to obtain.
Towards emotion prediction in spoken tutoring dialogues (2003)
Human tutors detect and respond to student emotional states, but current machine tutors do not. Our preliminary machine learning experiments involving transcription, emotion annotation and automatic...
A comparison of tutor and student behavior in speech versus text based tutoring (2003)
Carolyn P. Rosé, Diane Litman, Dumisizwe Bhembe, Kate Forbes, Scott Silliman, Ramesh Srivastava, ...
This paper describes preliminary work in exploring the relative effectiveness of speech versus text based tutoring. Most current tutorial dialogue systems are text based (Evens et
Recognizing emotions from student speech in tutoring dialogues (2003)
We investigate the automatic classification of student emotional states in a corpus of human-human spoken tutoring dialogues. We first annotated student turns in this corpus for negative, neutral and...
A comparison of tutor and student behavior in speech versus text based tutoring (2003)
Carolyn P. Rosé, Diane Litman, Dumisizwe Bhembe, Kate Forbes, Scott Silliman, Ramesh Srivastava, ...
This paper describes preliminary work in exploring the relative effectiveness of speech versus text based tutoring. Most current tutorial dialogue systems are text based (Evens et
Claire Cardie, Janyce Wiebe, Theresa Wilson, Diane Litman
While much recent progress has been made in research on fact-based question answering, our work aims to extend question-answering research in a different direction --- to handle multi-perspective...
Recognizing and organizing opinions expressed in the world press (2003)
Janyce Wiebe, Eric Breck, Chris Buckley, Claire Cardie, Paul Davis, Bruce Fraser, ...
Tomorrow’s question answering systems will need to have the ability to process information about beliefs, opinions, and evaluations—the perspective of an agent. Answers to many simple factual
Claire Cardie, Janyce Wiebe, Theresa Wilson, Diane Litman
While much recent progress has been made in research on fact-based question answering (Voorhees and Dice, 2000; Voorhees, 2001), our work aims to extend question-answering research in a different...
Recognizing and Organizing Opinions (2003)
Expressed In The, Janyce Wiebe, Eric Breck, Chris Buckley, Claire Cardie, Paul Davis, ...
This paper reports on an exploratory project investigating multiple perspectives in question answering (MPQA). The project was conducted as a summer workshop
Optimizing dialogue management with reinforcement learning: Experiments with the njfun system (2002)
Satinder Singh, Diane Litman, Michael Kearns, Marilyn Walker
Designing the dialogue policy of a spoken dialogue system involves many nontrivial choices. This paper presents a reinforcement learning approach for automatically optimizing a dialogue policy, which...
Optimizing dialogue management with reinforcement learning: Experiments with the njfun system (2002)
Satinder Singh, Diane Litman, Michael Kearns, Marilyn Walker
Designing the dialogue policy of a spoken dialogue system involves many nontrivial choices. This paper presents a reinforcement learning approach for automatically optimizing a dialogue policy, which...
Optimizing dialogue management with reinforcement learning: Experiments with the njfun system (2002)
Satinder Singh, Diane Litman, Michael Kearns, Marilyn Walker
Designing the dialogue policy of a spoken dialogue system involves many nontrivial choices. This paper presents a reinforcement learning approach for automatically optimizing a dialogue policy, which...
Predicting user reactions to system error (2001)
Diane Litman, Julia Hirschberg
diane/julia¡ This paper focuses on the analysis and prediction of so-called aware sites, defined as turns where a user of a spoken dialogue system first becomes aware that the system has made a...
Predicting user reactions to system error (2001)
Diane Litman, Julia Hirschberg
diane/julia¡ This paper focuses on the analysis and prediction of so-called aware sites, defined as turns where a user of a spoken dialogue system first becomes aware that the system has made a...
Abstract. The fields of user modeling and natural language processing have been closely linked since the early days of user modeling. Natural language systems consult user models in order to improve...
Natural Language Processing and User Modeling: Synergies and (2001)
Limitations Ingrid Zukerman, Ingrid Zukerman, Diane Litman
The fields of user modeling and natural language processing have been closely linked since the early days of user modeling. Natural language systems consult user models in order to improve their...
NJFun: A reinforcement learning spoken dialogue system (2000)
Diane Litman, Satinder Singh, Michael Kearns, Marilyn Walker
This paper describes NJFun, a real-time spoken dia-logue systemthat-provides users with information about things to d ~ in New Jersey. NJFun auto-matically optimizes its dialogue strategy over time,...
Towards developing general models of usability with (2000)
Marilyn Walker, Ace Kamm, Diane Litman
The design of methods for performmce evaluation is a major open resem'ch issue in the re'ca of spoken lmguage dialogue systems. In this paper we present the PARADISE methodology for...
Marilyn Walker, Irene Langkilde, Jerry Wright, Alien Gorin, Diane Litman
Current spoken dialogue systems are deficient in their strategies for preventing, identifying and re-pairing problems that arise in the conversation. This paper reports results on learning to...
Generalizing prosodic prediction of speech recognition errors (2000)
Julia Hirschberg, Diane Litman, Marc Swerts
Since users of spoken dialogue systems have difficulty correcting system misconceptions, it is important for automatic speech recognition (ASR) systems to know when their best hypothesis is...
Corrections in spoken dialogue systems (2000)
Marc Swerts, Diane Litman, Julia Hirschberg
This study analyzes user corrections of system errors in the TOOT spoken dialogue system. We find that corrections differ from noncorrections prosodically, in ways consistent with hyperarticulated...
Towards Developing General Models of Usability with PARADISE (2000)
Marilyn Walker, Candace Kamm, Ace Kamm, Diane Litman
The design of methods for performance evaluation is a major open research issue in the area of spoken language dialogue systems. This paper presents the PARADISE methodology for developing predictive...
Marilyn Walker, Irene Langkilde, Jerry Wright, Allen Gorin, Diane Litman
Current spoken dialogue systems are deficient in their strategies for preventing, identifying and repairing problems that arise in the conversation. This paper reports results on learning to...
Towards Developing General Models of Usability with PARADISE (2000)
Marilyn Walker, Candace Kamm, Ace Kamm, Diane Litman
The design of methods for performance evaluation is a major open research issue in the area of spoken language dialogue systems. In this paper we present the PARADISE methodology for developing...
Predicting Automatic Speech Recognition Performance Using Prosodic Cues (2000)
Julia Hirschberg, Diane Litman, Marc Swerts
In spoken dialogue systems, it is important for a system to know how likely a speech recognition hypothesis is to be correct, so it can reprompt for fresh input, or, in cases where many errors have...
Julia Hirschberg, Marilyn A, Irene Langkilde, Jerry Wright, Allen Gorin, ...
man, Kim, Ashok Kalyanswamy, Julie Silverman, Sara Basson, and Dina Yashchin. 1993. Synthesiser intelligibility in the context of a name-and-address information service. In Proceedings of...
Automatic Prediction of Problematic Human-Computer Dialogues in How May I Help You? (1999)
Langkilde, Irene, Walker, Marilyn A, Wright, Jerry, Gorin, Allen, Litman, Diane
Automatic prediction of problematic human-computer dialogues in How May I Help You (1999)
Irene Langkilde, Marilyn Walker, Jerry Wright, Allen Gorin, Diane Litman
We apply machine learning methods to automatically identify and predict problematic human-computer dialogues in a corpus collected during a wizard override trial of the How May I Help You task. The...
Automatic prediction of problematic human-computer dialogues in How May I Help You (1999)
Irene Langkilde, Marilyn Walker, Jerry Wright, Allen Gorin, Diane Litman
We report results on learning to automatically identify and predict problematic human-computer dialogues in a corpus collected during a wizard-override trial of the How May I Help You task. We show...
Reinforcement Learning for Spoken Dialogue Systems (1999)
Satinder Singh Att, Satinder Singh, Michael Kearns, Diane Litman, Marilyn Walker
Recently, a number of authors have proposed treating dialogue systems as Markov decision processes (MDPs). However, the practical application of MDP algorithms to dialogue systems faces a number of...
Reinforcement Learning for Spoken Dialogue Systems (1999)
Satinder Singh Att, Satinder Singh, Michael Kearns, Diane Litman, Marilyn Walker
Recently, a number of authors have proposed treating dialogue systems as Markov decision processes (MDPs). However, the practical application of MDP algorithms to dialogue systems faces a number of...
Reinforcement Learning for Spoken Dialogue Systems (1999)
Satinder Singh Att, Satinder Singh, Michael Kearns, Diane Litman, Marilyn Walker
Recently, a number of authors have proposed treating dialogue systems as Markov decision processes (MDPs). However, the practical application of MDP algorithms to dialogue systems faces a number of...
Reinforcement Learning for Spoken Dialogue Systems (1999)
Satinder Singh, Michael Kearns, Diane Litman, Marilyn Walker
Recently, a number of authors have proposed treating dialogue systems as Markov decision processes (MDPs). However, the practical application of MDP algorithms to dialogue systems faces a number of...
Automatic Prediction of Problematic Human-Computer Dialogue in 'How May I Help You?' (1999)
Irene Langkilde, Marilyn Walker, Jerry Wright, Allen Gorin, Diane Litman
We report results on learning to automatically identify and predict problematic human-computer dialogues in a corpus collected during a wizard-override trial of the How May I Help You task. We show...
Prosodic Cues to Recognition Errors (1999)
Julia Hirschberg, Diane Litman, Marc Swerts
We identify methods of distinguishing between correctly and incorrectly recognized utterances (scored by hand for semantic concept accuracy) for a speech recognition system, using acoustic/prosodic...
Photocopy of typescript.
Evaluating Response Strategies in a Web-Based Spoken Dialogue Agent (1998)
Diane Litman, Shimei Pan, Marilyn A. Walker
While the notion of a cooperative response has been the focus of considerable research in natural language dialogue systems, there has been little empirical work demonstrating how such responses lead...
Modeling Dynamic Collections of Interdependent Objects Using Path-Based Rules (1997)
Standard object-oriented languages do not provide language support for modeling changing collections of interdependent objects. We propose that R++, an integration of the rule and objectoriented...
Cue Phrase Classification Using Machine Learning (1996)
Cue phrases may be used in a discourse sense to explicitly signal discourse structure, but also in a sentential sense to convey semantic rather than structural information. Correctly classifying cue...
Cue Phrase Classification Using Machine Learning (1996)
Cue phrases may be used in a discourse sense to explicitly signal discourse structure, but also in a sentential sense to convey semantic rather than structural information. Correctly classifying cue...
R++: Using Rules in Object-Oriented Designs (1996)
Anil Mishra, Johannes P. Ros, Anoop Singhal, Gary Weiss, Diane Litman, ...
System requirements often express some of the behavior of a domain model in the form of policies, constraints, invariants, and rules. Although an object-oriented approach helps with the design of...
Using Rules in Object-Oriented Designs (1996)
Daniel Dvorak, Anil Mishra, Johannes P. Ros, Anoop Singhal, Gary Weiss, Diane Litman
System requirements often express some of the behavior of a domain model in the form of policies, constraints, invariants, and rules. Although an object-oriented approach helps with the design of...
Using Rules in Object-Oriented Designs (1996)
Daniel Dvorak, Anil Mishra, Johannes P. Ros, Anoop Singhal, Gary Weiss, Diane Litman
System requirements often express some of the behavior of a domain model in the form of policies, constraints, invariants, and rules. Although an object-oriented approach helps with the design of...
Subsumption and Recognition of Heterogeneous Constraint Networks (1994)
Terminological knowledge representation (tkr) systems such as kl-one are widely used in AI to construct concept taxonomies based on subsumption inferences. However, current tkr systems are unable to...
Empirical studies on the disambiguation of cue phrases (1993)
Julia Hirschberg, Diane Litman
Cue phrases are linguistic expressions such as now and well that function as explicit indicators of the structure of a discourse. For example, now may signal the beginning of a subtopic or a return...
Empirical Studies on the Disambiguation of Cue Phrases (1993)
Julia Hirschberg, Diane Litman
Cue phrases are linguistic expressions such as now and well that function as explicit indicators of the structure of a discourse. For example, now may signal the beginning of a subtopic or a return...
Terminological Reasoning with Constraint Networks and an Application to Plan Recognition (1992)
Terminological systems, such as KL-ONE and K-Rep, are widely used in AI to represent and reason with concept descriptions. They compute subsumption relations between concepts and automatically...