Transfer Learning for Image Classification with Sparse Prototype Representations (2009)
Ariadna Quattoni, Michael Collins, Trevor Darrell
To learn a new visual category from few examples, prior knowledge from unlabeled data as well as previous related categories may be useful. We develop a new method for transfer learning which...
Reducing Drift in Differential Tracking (2009)
We present methods for turning pair-wise registration algorithms into drift-free trackers. Such registration algorithms are abundant, but the simplest techniques for building trackers on top of them...
Konrad Tollmar, David Demirdjian, Trevor Darrell
The Problem: Physical and perceptual interfaces to games and virtual environments are an exciting new interface paradigm. Recent advances in computer vision make real-time sensing of users ’...
Abstract Localizing a Network of Non-Overlapping Cameras (2008)
We describe a method for recovering the extrinsic calibration parameters of non-overlapping cameras in a multi-camera system. Non-overlapping camera networks can cover a much wider area than...
Incorporating Object Tracking Feedback into Background Maintenance Framework (2008)
Leonid Taycher, Trevor Darrell
Adaptive background modeling/subtraction techniques are popular, in particular, because they are able to cope with background variations that are due to lighting variations. Unfortunately these...
Abstract Localizing a Network of Non-Overlapping Cameras (2008)
We describe a method for recovering the extrinsic calibration parameters of non-overlapping cameras in a multi-camera system. Non-overlapping camera networks can cover a much wider area than...
Learning Visual Representations using Images with Captions (2008)
Ariadna Quattoni, Michael Collins, Trevor Darrell
Current methods for learning visual categories work well when a large amount of labeled data is available, but can run into severe difficulties when the number of labeled examples is small. When...
Konrad Tollmar, David Demirdjian, Trevor Darrell
Physical and perceptual interfaces to games and virtual environments are an exciting new interface paradigm. Recent advances in computer vision make real-time sensing of users ’ position and pose...
Head Gestures for Perceptual Interfaces: The Role of Context in Improving Recognition (2008)
Head pose and gesture offer several conversational grounding cues and are used extensively in face-to-face interaction among people. To recognize visual feedback efficiently, humans often use...
Rank Priors for Continuous Non-Linear Dimensionality Reduction (2008)
Geiger, Andreas, Urtasun, Raquel, Darrell, Trevor, Stiefelhagen, Rainer
Non-linear dimensionality reduction methods are powerful techniques todeal with high-dimensional datasets. However, they often aresusceptible to local minima and perform poorly when initialized...
Rank Priors for Continuous Non-Linear Dimensionality Reduction (2008)
Geiger, Andreas, Urtasun, Raquel, Darrell, Trevor, Stiefelhagen, Rainer
Non-linear dimensionality reduction methods are powerful techniques todeal with high-dimensional datasets. However, they often aresusceptible to local minima and perform poorly when initialized...
Combining Object and Feature Dynamics in Probabilistic Tracking (2008)
Leonid Taycher, Trevor Darrell
Objects can exhibit different dynamics at different scales, and this is often exploited by visual tracking algorithms. A local dynamic model is typically used to extract image features that are then...
A Projected Subgradient Method for Scalable Multi-Task Learning (2008)
Quattoni, Ariadna, Carreras, Xavier, Collins, Michael, Darrell, Trevor
Recent approaches to multi-task learning have investigated the use of a variety of matrix norm regularization schemes for promoting feature sharing across tasks.In essence, these approaches aim at...
A Projected Subgradient Method for Scalable Multi-Task Learning (2008)
Quattoni, Ariadna, Carreras, Xavier, Collins, Michael, Darrell, Trevor
Recent approaches to multi-task learning have investigated the use of a variety of matrix norm regularization schemes for promoting feature sharing across tasks.In essence, these approaches aim at...
Fast concurrent object classification and localization (2008)
Yeh, Tom, Lee, John J., Darrell, Trevor
Object localization and classification are important problems incomputer vision. However, in many applications, exhaustive searchover all class labels and image locations is...
Fast concurrent object classification and localization (2008)
Yeh, Tom, Lee, John J., Darrell, Trevor
Object localization and classification are important problems incomputer vision. However, in many applications, exhaustive searchover all class labels and image locations is...
Transferring Nonlinear Representations using Gaussian Processes with a Shared Latent Space (2008)
Urtasun, Raquel, Quattoni, Ariadna, Lawrence, Neil, Darrell, Trevor
When a series of problems are related, representations derived from learning earlier tasks may be useful in solving later problems. In this paper we propose a novel approach to transfer learning with...
Transferring Nonlinear Representations using Gaussian Processes with a Shared Latent Space (2008)
Urtasun, Raquel, Quattoni, Ariadna, Lawrence, Neil, Darrell, Trevor
When a series of problems are related, representations derived from learning earlier tasks may be useful in solving later problems. In this paper we propose a novel approach to transfer learning with...
Perceptual Multimodal Interfaces Perceptive Presence (2008)
Frank Bentley, Konrad Tollmar, David Demirdjian, Kimberle Koile, Trevor Darrell
Ever since computers were first networked together, people have used them for communication. From email—one of the first network applications—to the original MIT Zephyr notification system1 to...
Vision-Aided Acoustic Array Processing for Perceptive Environments (2008)
The Problem: We seek to create a system that integrates microphone arrays and cameras for use in perceptive environments. The goal is to extract low-noise signals from one or more sound sources in...
Modeling Interactive Agents in ALIVE (2008)
Pattie Maes, Bruce Blumberg, Trevor Darrell, Alex Pentl, Alan Wexelblat
In this video we discuss the design and implementation of a novel system which allows wireless full-body interaction between a human participant and a graphical world inhabited by autonomous agents....
Object Category Recognition Using Probabilistic Fusion of Speech and Image Classifiers (2008)
Abstract. Multimodal scene understanding is an integral part of humanrobot interaction (HRI) in situated environments. Especially useful is category-level recognition, where the the system can...
Conditional Sequence Model for Context-based Recognition of Gaze Aversion (2008)
Abstract. Eye gaze and gesture form key conversational grounding cues that are used extensively in face-to-face interaction among people. To accurately recognize visual feedback during interaction,...
Head Gestures for Perceptual Interfaces: The Role of Context in Improving Recognition (2008)
Head pose and gesture offer several conversational grounding cues and are used extensively in face-to-face interaction among people. To recognize visual feedback efficiently, humans often use...
Person Tracking with Stereo Range Sensors (2008)
John Viloria, David Demirdjian, Neal Checka, Trevor Darrell
The Problem: The goal of this project is to design and build a person tracking system using stereo cameras. Our system, which stands in an ordinary conference room, can track multiple people and...
Gregory Shakhnarovich, Piotr Indyk, Trevor Darrell
The nearest-neighbor (NN) problem occurs in the literature under many names, including the best match or the post office problem. The problem is of significant importance to several areas of computer...
Range Segmentation Using Visibility Constraints (2008)
Leonid Taycher, Trevor Darrell
The Problem: We are looking to produce a foreground segmentation mechanism that would work under widely varying illumination conditions. Motivation: One of the most important tasks of computer vision...
Transfer learning for image classification with sparse prototype representations (2008)
Quattoni, Ariadna, Collins, Michael, Darrell, Trevor
To learn a new visual category from few examples, prior knowledge from unlabeled data as well as previous related categories may be useful. We develop a new method for transfer learning which...
Transfer learning for image classification with sparse prototype representations (2008)
Quattoni, Ariadna, Collins, Michael, Darrell, Trevor
To learn a new visual category from few examples, prior knowledge from unlabeled data as well as previous related categories may be useful. We develop a new method for transfer learning which...
Unsupervised Distributed Feature Selection for Multi-view Object Recognition (2008)
Christoudias, C. Mario, Urtasun, Raquel, Darrell, Trevor
Object recognition accuracy can be improved when information frommultiple views is integrated, but information in each view can oftenbe highly redundant. We consider the problem of distributed...
Unsupervised Distributed Feature Selection for Multi-view Object Recognition (2008)
Christoudias, C. Mario, Urtasun, Raquel, Darrell, Trevor
Object recognition accuracy can be improved when information frommultiple views is integrated, but information in each view can oftenbe highly redundant. We consider the problem of distributed...
Combining Object and Feature Dynamics in Probabilistic Tracking (2008)
Leonid Taycher, Trevor Darrell
Objects can exhibit different dynamics at different scales, and this is often exploited by visual tracking algorithms. A local dynamic model is typically used to extract image features that are then...
William T. Freeman, Trevor Darrell, Paul Viola
People can understand complex auditory and visual information, often using one to disambiguate the other. Automated analysis, even at a lowlevel, faces severe challenges, including the lack of...
Kristen Grauman, Trevor Darrell
Pyramid intersection is an efficient method for computing an approximate partial matching between two sets of feature vectors. We introduce a novel pyramid embedding based on a hierarchy of...
MULTIMODAL COMMUNICATION ERROR DETECTION FOR DRIVER-CAR INTERACTION (2008)
Sy Bor Wang, David Demirdjian, Trevor Darrell, Hedvig Kjellström
Abstract: Speech recognition systems are now used in a wide variety of domains. They have recently been introduced in cars for hand-free control of radio, cell-phone and navigation applications....
Incorporating Object Tracking Feedback into Background Maintenance Framework (2008)
Leonid Taycher, Trevor Darrell
Adaptive background modeling/subtraction techniques are popular, in particular, because they are able to cope with background variations that are due to lighting variations. Unfortunately these...
Hidden Conditional Random Fields for Gesture Recognition (2008)
Sy Bor, Wang Ariadna, Morency David Demirdjian, Trevor Darrell
We introduce a discriminative hidden-state approach for the recognition of human gestures. Gesture sequences often have a complex underlying structure, and models that can incorporate hidden...
1 Learning Embeddings for Fast Approximate Nearest Neighbor Retrieval................... vii (2008)
Gregory Shakhnarovich, Trevor Darrell, Piotr Indyk, Gregory Shakhnarovich, Trevor Darrell, Piotr Indyk, ...
ii
Ali Rahimi, Ben Recht, Trevor Darrell
Many high-dimensional time-varying signals can be modeled as a sequence of noisy nonlinear observations of a low-dimensional dynamical process. Given high-dimensional observations and a distribution...
Learning visual representations using images with captions (2008)
Ariadna Quattoni, Michael Collins, Trevor Darrell
Current methods for learning visual categories work well when a large amount of labeled data is available, but can run into severe difficulties when the number of labeled examples is small. When...
Gregory Shakhnarovich, Advisor Prof, Trevor Darrell, Gregory Shakhnarovich
estimation for visual objects. ⋄ Machine learning, in particular example-based methods, unsupervised and semi-supervised learning. ⋄ Brain-machine interfaces, in particular neural decoding and...
Segmentation of Rigidly Moving Objects using Multiple Kalman Filters (2007)
Trevor Darrell, Ali Azarbayejani, Alex P. Pentl
In this paper we describe a method for structure-from-motion recovery when there are multiple objects in the scene. We use our recently developed recursive estimation technique to recover shape and...
Inferring Statistical Model 3D Structure with a Image-Based Shape (2007)
Kristen Grauman, Gregory Shakhnarovich, Trevor Darrell
We present an image-based approach to infer 3D structure parameters using a probabilistic "shape+structure " model. The 3D shape of a class of objects may be represented by sets of...
Chris Mario Christoudias, Louis-philip Morency, Trevor Darrell
Statistical shape and texture appearance models are powerful image representations, but previously had been restricted to 2D or simple 3D shapes. In this paper we present a novel 3D morphable model...
Range Segmentation Using Visibility Constraints (2007)
Leonid Taycher, Trevor Darrell
Visibility constraints can aid the segmentation of foreground objects in a scene observed with multiple range imagers. Points may be labeled as foreground if they can be determined to occlude some...
Pattie Maes, Trevor Darrell, Bruce Blumberg, Alex Pentl
The cumbersomenature of wired interfaces often limits the range of application of virtual environments. In this paper we discuss the design and implementation of a novel system, called ALIVE, which...
William T. Freeman, Trevor Darrell, Paul Viola
People can understand complex auditory and visual information, often using one to disambiguate the other. Automated analysis, even at a lowlevel, faces severe challenges, including the lack of...
Gregory Shakhnarovich, Paul Viola, Trevor Darrell
Example-based methods are effective for parameter estimation problems when the underlying system is simple or the dimensionality of the input is low. For complex and high-dimensional problems such as...
Wide Web Consortium's Technology and Society activities. (2007)
Mark Ackerman, Trevor Darrell, Daniel J. Weitzner
Oxygen and works on collaborative systems. He is also an associate professor in
Kristen Grauman, Gregory Shakhnarovich, Trevor Darrell
We present a Bayesian approach to image-based visual hull reconstruction. The 3-D shape of an object of a known class is represented by sets of silhouette views simultaneously observed from multiple...
Transfering Nonlinear Representations using Gaussian Processes with a Shared Latent Space (2007)
Urtasun, Raquel, Quattoni, Ariadna, Darrell, Trevor
When a series of problems are related, representations derived fromlearning earlier tasks may be useful in solving later problems. Inthis paper we propose a novel approach to transfer learning...
Transfering Nonlinear Representations using Gaussian Processes with a Shared Latent Space (2007)
Urtasun, Raquel, Quattoni, Ariadna, Darrell, Trevor
When a series of problems are related, representations derived fromlearning earlier tasks may be useful in solving later problems. Inthis paper we propose a novel approach to transfer learning...
Learning to Transform Time Series with a Few Examples (2007)
Rahimi, Ali, Recht, Benjamin, Darrell, Trevor
We describe a semi-supervised regression algorithm that learns to transform one time series into another time series given examples of the transformation. This algorithm is applied to tracking, where...
Discriminative Gaussian Process Latent Variable Model for Classification (2007)
Urtasun, Raquel, Darrell, Trevor
Supervised learning is difficult with high dimensional input spacesand very small training sets, but accurate classification may bepossible if the data lie on a low-dimensional manifold....
Discriminative Gaussian Process Latent Variable Model for Classification (2007)
Urtasun, Raquel, Darrell, Trevor
Supervised learning is difficult with high dimensional input spacesand very small training sets, but accurate classification may bepossible if the data lie on a low-dimensional manifold....
Latent-Dynamic Discriminative Models for Continuous Gesture Recognition (2007)
Morency, Louis-Philippe, Quattoni, Ariadna, Darrell, Trevor
Many problems in vision involve the prediction of a class label for each frame in an unsegmented sequence. In this paper we develop a discriminative framework for simultaneous sequence segmentation...
Latent-Dynamic Discriminative Models for Continuous Gesture Recognition (2007)
Morency, Louis-Philippe, Quattoni, Ariadna, Darrell, Trevor
Many problems in vision involve the prediction of a class label for each frame in an unsegmented sequence. In this paper we develop a discriminative framework for simultaneous sequence segmentation...
Learning Visual Representations using Images with Captions (2007)
Ariadna Quattoni, Michael Collins, Trevor Darrell
Current methods for learning visual categories work well when a large amount of labeled data is available, but can run into severe difficulties when the number of labeled examples is small. When...
The pyramid match kernel: Efficient learning with sets of features (2007)
Kristen Grauman, Trevor Darrell, Pietro Perona
In numerous domains it is useful to represent a single example by the set of the local features or parts that comprise it. However, this representation poses a challenge to many conventional machine...
Discriminative Gaussian Process Latent Variable Models for Classification (2007)
Raquel Urtasun, Trevor Darrell
Supervised learning is difficult with high dimensional input spaces and very small training sets, but accurate classification may be possible if the data lie on a low-dimensional manifold. Gaussian...
Transfering Nonlinear Representations using Gaussian Processes with a Shared Latent Space (2007)
Raquel Urtasun, Ariadna Quattoni, Trevor Darrell, Raquel Urtasun, Ariadna Quattoni, Trevor Darrell
When a series of problems are related, representations derived from learning earlier tasks may be useful in solving later problems. In this paper we propose a novel approach to transfer learning with...
Approximate Correspondences in High Dimensions (2006)
Grauman, Kristen, Darrell, Trevor
Pyramid intersection is an efficient method for computing an approximate partial matching between two sets of feature vectors. We introduce a novel pyramid embedding based on a hierarchy of...
Approximate Correspondences in High Dimensions (2006)
Grauman, Kristen, Darrell, Trevor
Pyramid intersection is an efficient method for computing an approximate partial matching between two sets of feature vectors. We introduce a novel pyramid embedding based on a hierarchy of...
Pyramid Match Kernels: Discriminative Classification with Sets of Image Features (version 2) (2006)
Grauman, Kristen, Darrell, Trevor
Discriminative learning is challenging when examples are sets of features, and the sets vary in cardinality and lack any sort of meaningful ordering. Kernel-based classification methods can learn...
Pyramid Match Kernels: Discriminative Classification with Sets of Image Features (version 2) (2006)
Grauman, Kristen, Darrell, Trevor
Discriminative learning is challenging when examples are sets of features, and the sets vary in cardinality and lack any sort of meaningful ordering. Kernel-based classification methods can learn...
Kevin W. Wilson, Student Member, Trevor Darrell
Abstract—Speech source localization in reverberant environments has proved difficult for automated microphone array systems. Because of its nonstationary nature, certain features observable in the...
Unsupervised learning of categories from sets of partially matching image features (2006)
Kristen Grauman, Trevor Darrell
We present a method to automatically learn object categories from unlabeled images. Each image is represented by an unordered set of local features, and all sets are embedded into a space where they...
Learning to Transform Time Series with a Few (2006)
Ali Rahimi, Ben Recht, Trevor Darrell
We describe a semi-supervised regression algorithm that learns to transform one time series into another time series given examples of the transformation. This algorithm is applied to tracking, where...
Conditional random people: Tracking humans with crfs and grid filters (2006)
Leonid Taycher, Leonid Taycher, Gregory Shakhnarovich, Gregory Shakhnarovich, David Demirdjian, David Demirdjian, ...
We describe a state-space tracking approach based on a Conditional Random Field (CRF) model, where the observation potentials are learned from data. We find functions that embed both state and...
Approximate correspondences in high dimensions (2006)
Kristen Grauman, Kristen Grauman, Trevor Darrell, Trevor Darrell
Pyramid intersection is an efficient method for computing an approximate partial matching between two sets of feature vectors. We introduce a novel pyramid embedding based on a hierarchy of...
Conditional Random People: Tracking Humans with CRFs and Grid Filters (2005)
Taycher, Leonid, Shakhnarovich, Gregory, Demirdjian, David, Darrell, Trevor
We describe a state-space tracking approach based on a Conditional Random Field(CRF) model, where the observation potentials are \emph{learned} from data. Wefind functions that embed both state and...
Conditional Random People: Tracking Humans with CRFs and Grid Filters (2005)
Taycher, Leonid, Shakhnarovich, Gregory, Demirdjian, David, Darrell, Trevor
We describe a state-space tracking approach based on a Conditional Random Field(CRF) model, where the observation potentials are \emph{learned} from data. Wefind functions that embed both state and...
Nonlinear Latent Variable Models for Video Sequences (2005)
Rahimi, Ali, Recht, Ben, Darrell, Trevor
Many high-dimensional time-varying signals can be modeled as a sequence of noisy nonlinear observations of a low-dimensional dynamical process. Given high-dimensional observations and a distribution...
Nonlinear Latent Variable Models for Video Sequences (2005)
Rahimi, Ali, Recht, Ben, Darrell, Trevor
Many high-dimensional time-varying signals can be modeled as a sequence of noisy nonlinear observations of a low-dimensional dynamical process. Given high-dimensional observations and a distribution...
Pyramid Match Kernels: Discriminative Classification with Sets of Image Features (2005)
Grauman, Kristen, Darrell, Trevor
Discriminative learning is challenging when examples are setsof local image features, and the sets vary in cardinality and lackany sort of meaningful ordering. Kernel-based classificationmethods can...
Pyramid Match Kernels: Discriminative Classification with Sets of Image Features (2005)
Grauman, Kristen, Darrell, Trevor
Discriminative learning is challenging when examples are setsof local image features, and the sets vary in cardinality and lackany sort of meaningful ordering. Kernel-based classificationmethods can...
Combining Object and Feature Dynamics in Probabilistic Tracking (2005)
Taycher, Leonid, Fisher III, John W., Darrell, Trevor
Objects can exhibit different dynamics at different scales, a property that isoftenexploited by visual tracking algorithms. A local dynamicmodel is typically used to extract image features that are...
Combining Object and Feature Dynamics in Probabilistic Tracking (2005)
Taycher, Leonid, Fisher III, John W., Darrell, Trevor
Objects can exhibit different dynamics at different scales, a property that isoftenexploited by visual tracking algorithms. A local dynamicmodel is typically used to extract image features that are...
The pyramid match kernel: Discriminative classification with sets of image features (2005)
Kristen Grauman, Trevor Darrell
Discriminative learning is challenging when examples are sets of features, and the sets vary in cardinality and lack any sort of meaningful ordering. Kernel-based classification methods can learn...
Production domain modeling of pronunciation for visual speech recognition (2005)
Kate Saenko, Karen Livescu, James Glass, Trevor Darrell
Articulatory feature models have been proposed in the automatic speech recognition community as an alternative to phone-based models of speech. In this paper, we extend this approach to the visual...
Avoiding the streetlight effect: Tracking by exploring likelihood modes (2005)
David Demirdjian, Leonid Taycher, Gregory Shakhnarovich, Kristen Grauman, Trevor Darrell
Classic methods for Bayesian inference effectively constrain search to lie within regions of significant probability of the temporal prior. This is efficient with an accurate dynamics model, but...
The pyramid match kernel: Discriminative classification with sets of image features (2005)
Kristen Grauman, Trevor Darrell
Discriminative learning is challenging when examples are sets of features, and the sets vary in cardinality and lack any sort of meaningful ordering. Kernel-based classification methods can learn...
Visual speech recognition with loosely synchronized feature streams (2005)
Kate Saenko, Karen Livescu, Michael Siracusa, Kevin Wilson, James Glass, Trevor Darrell
We present an approach to detecting and recognizing spoken isolated phrases based solely on visual input. We adopt an architecture that first employs discriminative detection of visual speech and...
Abstract Combining object and feature dynamics in probabilistic tracking (2005)
Leonid Taycher, Trevor Darrell
Objects can exhibit different dynamics at different spatio-temporal scales, a property that is often exploited by visual tracking algorithms. A local dynamic model is typically used to extract image...
Efficient image matching with distributions of local invariant features (2005)
Kristen Grauman, Trevor Darrell
Sets of local features that are invariant to common image transformations are an effective representation to use when comparing images; current methods typically judge feature sets ’ similarity via...
A picture is worth a thousand Keywords: image –based object search on a mobile platform (2005)
Tom Yeh, Kristen Grauman, Konrad Tollmar, Trevor Darrell
Finding information based on an object’s visual appearance is useful when specific keywords for the object are not known. We have developed a mobile image-based search system that takes images of...
Mitsubishi Electric Research Laboratories (2005)
Http Www Merl, Mit Csail, Candace Sidner, Christopher Lee, Trevor Darrell
Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how dialog context from an embodied...
Contextual Recognition of Head Gestures (2005)
Ace L. Sidner, Candace Sidner, Christopher Lee, Christopher Lee, Trevor Darrell, Trevor Darrell
Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how dialog context from an embodied...
Towards contextbased visual feedback Recognition for Embodied Agents (2005)
Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how contextural information can improve visual...
– Distribution-based – Neural-network based – Boosting based (2005)
Bill Freeman Mit, Trevor Darrell, Paul Viola
Photos of class • What makes detection easy or hard? • What makes recognition easy or hard? E5 class, and recognition machine
Nonlinear Latent Variable Models for Video Sequences (2005)
Ali Rahimi, Ben Recht, Trevor Darrell
Many high-dimensional time-varying signals can be modeled as a sequence of noisy nonlinear observations of a low-dimensional dynamical process. Given high-dimensional observations and a distribution...
Towards contextbased visual feedback Recognition for Embodied Agents (2005)
During face-to-face conversation, people use visual feedback (e.g., head and eye gesture) to communicate relevant information and to synchronize rhythm between participants. When recognizing visual...
Combining Object and Feature Dynamics in Probabilistic Tracking (2005)
Leonid Taycher, Trevor Darrell, Leonid Taycher, Trevor Darrell
Objects can exhibit different dynamics at different scales, a property that is often exploited by visual tracking algorithms. A local dynamic model is typically used to extract image features that...
A picture is worth a thousand Keywords: image –based object search on a mobile platform (2005)
Tom Yeh, Kristen Grauman, Konrad Tollmar, Trevor Darrell
Finding information based on an object’s visual appearance is useful when specific keywords for the object are not known. We have developed a mobile image-based search system that takes images of...
The pyramid match kernel: Discriminative classification with sets of image features (2005)
Kristen Grauman, Trevor Darrell
Discriminative learning is challenging when examples are sets of local image features, and the sets vary in cardinality and lack any sort of meaningful ordering. Kernel-based classification methods...
Visual speech recognition with loosely synchronized feature streams (2005)
Kate Saenko, Karen Livescu, Michael Siracusa, Kevin Wilson, James Glass, Trevor Darrell
We present an approach to detecting and recognizing spoken isolated phrases based solely on visual input. We adopt an architecture that first employs discriminative detection of visual speech and...
Efficient Image Matching with Distributions of Local Invariant Features (2004)
Grauman, Kristen, Darrell, Trevor
Sets of local features that are invariant to common image transformations are an effective representation to use when comparing images; current methods typically judge feature sets' similarity via a...
Efficient Image Matching with Distributions of Local Invariant Features (2004)
Grauman, Kristen, Darrell, Trevor
Sets of local features that are invariant to common image transformations are an effective representation to use when comparing images; current methods typically judge feature sets' similarity via a...
Virtual Visual Hulls: Example-Based 3D Shape Estimation from a Single Silhouette (2004)
Grauman, Kristen, Shakhnarovich, Gregory, Darrell, Trevor
Recovering a volumetric model of a person, car, or other object of interest from a single snapshot would be useful for many computer graphics applications. 3D model estimation in general is hard, and...
Virtual Visual Hulls: Example-Based 3D Shape Estimation from a Single Silhouette (2004)
Grauman, Kristen, Shakhnarovich, Gregory, Darrell, Trevor
Recovering a volumetric model of a person, car, or other objectof interest from a single snapshot would be useful for many computergraphics applications. 3D model estimation in general is hard,...
Virtual Visual Hulls: Example-Based 3D Shape Estimation from a Single Silhouette (2004)
Grauman, Kristen, Shakhnarovich, Gregory, Darrell, Trevor
Recovering a volumetric model of a person, car, or other objectof interest from a single snapshot would be useful for many computergraphics applications. 3D model estimation in general is hard,...
Virtual Visual Hulls: Example-Based 3D Shape Estimation from a Single Silhouette (2004)
Grauman, Kristen, Shakhnarovich, Gregory, Darrell, Trevor
Recovering a volumetric model of a person, car, or other object of interest from a single snapshot would be useful for many computer graphics applications. 3D model estimation in general is hard, and...
Fast contour matching using approximate earth mover’s distance (2004)
Kristen Grauman, Trevor Darrell
Weighted graph matching is a good way to align a pair of shapes represented by a set of descriptive local features; the set of correspondences produced by the minimum cost matching between two shapes...
Fast contour matching using approximate earth mover’s distance (2004)
Kristen Grauman, Trevor Darrell
Weighted graph matching is a good way to align a pair of shapes represented by a set of descriptive local features; the set of correspondences produced by the minimum cost matching between two shapes...
Speaker association with signal-level audiovisual fusion (2004)
John W. Fisher, Trevor Darrell
Abstract—Audio and visual signals arriving from a common source are detected using a signal-level fusion technique. A probabilistic multimodal generation model is introduced and used to derive an...
Combining Simple Models to Approximate Complex Dynamics (2004)
Leonid Taycher, Trevor Darrell
Stochastic tracking of structured models in monolithic state spaces often requires modeling complex distributions that are difficult to represent with either parametric or sample-based approaches. We...
Mitsubishi Electric Research Laboratories (2004)
Http Www Merl, Christopher Lee Merl, Neal Lesh Merl, Ace L. Sidner, Trevor Darrell, Mit Vision Group, ...
In this demo we describe our ongoing efforts to build a robot that can collaborate with a person in hosting activities. We illustrate our current robot's conversations, which include gestures of...
Mitsubishi Electric Research Laboratories (2004)
Http Www Merl, Candace L. Sidner, Trevor Darrell
This paper has been accepted for presentation at the AISB symposium on Conversational Informatics, Hatfield, England, April, 2005
Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. While the machine interpretation of these cues has previously...
Conditional random fields for object recognition (2004)
Ariadna Quattoni, Michael Collins, Trevor Darrell
We present a discriminative part-based approach for the recognition of object classes from unsegmented cluttered scenes. Objects are modeled as flexible constellations of parts conditioned on local...
Nodding in conversations with a robot (2004)
Ashish Kapoor, Trevor Darrell, Christopher Lee, Christopher Lee, Neal Lesh, Neal Lesh, ...
In this demo we describe our ongoing efforts to build a robot that can collaborate with a person in hosting activities. We illustrate our current robot´s conversations, which include gestures of...
IDeixis - Image-Based Deixis for Finding Location-Based Information (2004)
Konrad Tollmar, Tom Yeh, Trevor Darrell
Abstract. In this paper we describe an image-based approach to finding location-based information from camera-equipped mobile devices. We introduce a point-by-photograph paradigm, where users can...
Vision for Multimodal Conversational Interfaces (2003)
A classic goal for human computer interface is the conversational computer, which can freely interact with users through natural dialog. Advances in speech and language processing have made systems...
Fast Contour Matching Using Approximate Earth Mover's Distance (2003)
Grauman, Kristen, Darrell, Trevor
Weighted graph matching is a good way to align a pair of shapes represented by a set of descriptive local features; the set of correspondences produced by the minimum cost of matching features from...
Fast Contour Matching Using Approximate Earth Mover's Distance (2003)
Grauman, Kristen, Darrell, Trevor
Weighted graph matching is a good way to align a pair of shapesrepresented by a set of descriptive local features; the set ofcorrespondences produced by the minimum cost of matching features fromone...
Fast Contour Matching Using Approximate Earth Mover's Distance (2003)
Grauman, Kristen, Darrell, Trevor
Weighted graph matching is a good way to align a pair of shapesrepresented by a set of descriptive local features; the set ofcorrespondences produced by the minimum cost of matching features fromone...
Fast Contour Matching Using Approximate Earth Mover's Distance (2003)
Grauman, Kristen, Darrell, Trevor
Weighted graph matching is a good way to align a pair of shapes represented by a set of descriptive local features; the set of correspondences produced by the minimum cost of matching features from...
Activity Zones for Context-Aware Computing (2003)
Koile, Kimberle, Tollmar, Konrad, Demirdjian, David, Shrobe, Howard, Darrell, Trevor
Location is a primary cue in many context-aware computing systems, and is often represented as a global coordinate, room number, or Euclidean distance various landmarks. A user?s concept of location,...
Activity Zones for Context-Aware Computing (2003)
Koile, Kimberle, Tollmar, Konrad, Demirdjian, David, Shrobe, Howard, Darrell, Trevor
Location is a primary cue in many context-aware computing systems, and is often represented as a global coordinate, room number, or Euclidean distance various landmarks. A user?s concept of location,...
Fast Pose Estimation with Parameter Sensitive Hashing (2003)
Shakhnarovich, Gregory, Viola, Paul, Darrell, Trevor
Example-based methods are effective for parameter estimation problems when the underlying system is simple or the dimensionality of the input is low. For complex and high-dimensional problems such as...
Light Field Morphable Models (2003)
Christoudias, Chris Mario, Morency, Louis-Philippe, Darrell, Trevor
Statistical shape and texture appearance models are powerful image representations, but previously had been restricted to 2D or simple 3D shapes. In this paper we present a novel 3D morphable model...
Light Field Morphable Models (2003)
Christoudias, Chris Mario, Morency, Louis-Philippe, Darrell, Trevor
Statistical shape and texture appearance models are powerful image representations, but previously had been restricted to 2D or simple 3D shapes. In this paper we present a novel 3D morphable model...
Fast Pose Estimation with Parameter Sensitive Hashing (2003)
Shakhnarovich, Gregory, Viola, Paul, Darrell, Trevor
Example-based methods are effective for parameter estimation problems when the underlying system is simple or the dimensionality of the input is low. For complex and high-dimensional problems such as...
Inferring 3D Structure with a Statistical Image-Based Shape Model (2003)
Grauman, Kristen, Shakhnarovich, Gregory, Darrell, Trevor
We present an image-based approach to infer 3D structure parameters using a probabilistic "shape+structure'' model. The 3D shape of a class of objects may be represented by sets of contours from...
Inferring 3D Structure with a Statistical Image-Based Shape Model (2003)
Grauman, Kristen, Shakhnarovich, Gregory, Darrell, Trevor
We present an image-based approach to infer 3D structure parameters using a probabilistic "shape+structure'' model. The 3D shape of a class of objects may be represented by sets of contours from...
Inferring 3D Structure with a Statistical Image-Based Shape Model (2003)
Kristen Grauman, Gregory Shakhnarovich, Trevor Darrell
We present an image-based approach to infer 3D structure parameters using a probabilistic “shape+structure ” model. The 3D shape of a class of objects may be represented by sets of contours from...
Fast pose estimation with parameter-sensitive hashing (2003)
Gregory Shakhnarovich, Paul Viola, Trevor Darrell
Example-based methods are effective for parameter estimation problems when the underlying system is simple or the dimensionality of the input is low. For complex and high-dimensional problems such as...
Inferring 3D Structure with a Statistical Image-Based Shape Model (2003)
Kristen Grauman, Gregory Shakhnarovich, Trevor Darrell
We present an image-based approach to infer 3D structure parameters using a probabilistic “shape+structure ” model. The 3D shape of an object class is represented by sets of contours from...
A Bayesian Approach to Image-Based Visual Hull Reconstruction (2003)
Kristen Grauman, Gregory Shakhnarovich, Trevor Darrell
We present a Bayesian approach to image-based visual hull reconstruction. The 3-D shape of an object of a known class is represented by sets of silhouette views simultaneously observed from multiple...
Fast pose estimation with parameter sensitive hashing (2003)
Gregory Shakhnarovich, Paul Viola, Trevor Darrell
Example-based methods are effective for parameter estimation problems when the underlying system is simple or the dimensionality of the input is low. For complex and high-dimensional problems such as...
and 3D Structure Inference (2003)
Kristen Lorraine Grauman, Trevor Darrell, Kristen Lorraine Grauman
Fast pose estimation with parameter-sensitive hashing (2003)
Gregory Shakhnarovich, Paul Viola, Trevor Darrell
Example-based methods are effective for parameter estimation problems when the underlying system is simple or the dimensionality of the input is low. For complex and high-dimensional problems such as...
Adaptive view-based appearance model (2003)
We present a method for online rigid object tracking using an adaptive view-based appearance model. When the object’s pose trajectory crosses itself, our tracker has bounded drift and can track...
Fast pose estimation with parameter sensitive hashing (2003)
Gregory Shakhnarovich, Paul Viola, Trevor Darrell
Example-based methods ' are effective for parameter estimation problems when the underlying system is ' simple or the dimensionality of the input is ' low. For complex and...
Activity Zones for Context-Aware Computing (2003)
Kimberle Koile, Konrad Tollmar, David Demirdjian, Howard Shrobe, Trevor Darrell
Abstract. Location is a primary cue in many context-aware computing systems, and is often represented as a global coordinate, room number, or a set of Euclidean distances to various landmarks. A...
A Multi-Modal Approach for Determining (2003)
Speaker Location And, Michael Siracusa, Kevin Wilson, John Fisher, Trevor Darrell
This paper presents a multi-modal approach to locate a speaker in a scene and determine to whom he or she is speaking. We present a simple probabilistic framework that combines multiple cues derived...
Activity Zones for Context-Aware Computing (2003)
Kimberle Koile Konrad, Konrad Tollmar, David Demirdjian, Howard Shrobe, Trevor Darrell
Location is a primary cue in many context-aware computing systems, and is often represented as a global coordinate, room number, or a set of Euclidean distances to various landmarks. A user's...
Pose Estimation Using 3D View-Based Eigenspaces (2003)
Louis-Philippe Morency, Patrik Sundberg, Trevor Darrell
In this paper we present a method for estimating the absolute pose of a rigid object based on intensity and depth viewbased eigenspaces, built across multiple views of example objects of the same...
Untethered Gesture Acquisition and Recognition for Virtual World Manipulation (2003)
David Demirdjian, Teresa Ko, Trevor Darrell
Humans use a combination of gesture and speech to interact with objects and usually do so more naturally without holding a device or pointer. We present a system that incorporates user body-pose...
Fast pose estimation with parameter sensitive hashing (2003)
Gregory Shakhnarovich, Paul Viola, Trevor Darrell
Example-based methods are effective for parameter estimation problems when the underlying system is simple or the dimensionality of the input is low. For complex and high-dimensional problems such as...
Reaching for Dexterous Manipulation MIT EECS Area Exam (2003)
Exam Committee, Rodney Brooks, Leslie Kaelbling, Trevor Darrell, Push Singh
I review three papers in the area of dexterous manipulation by machines. These papers focus on different stages of the process of grasping an object. The first describes a method for generating a...
Bayesian Articulated Tracking Using Single Frame Pose Sampling (2003)
Leonid Taycher, Trevor Darrell
We propose a novel probabilistic tracking framework for articulated bodies that incorporates direct estimation of the pose posterior distribution. We derive a single frame articulated pose sampler,...
Exploring Vision-Based Interfaces: How to Use Your Head in Dual Pointing Tasks (2002)
Darrell, Trevor, Checka, Neal, Oh, Alice, Morency, Louis-Philippe
The utility of vision-based face tracking for dual pointing tasks is evaluated. We first describe a 3-D face tracking technique based on real-time parametric motion-stereo, which is non-invasive,...
Exploring Vision-Based Interfaces: How to Use Your Head in Dual Pointing Tasks (2002)
Darrell, Trevor, Checka, Neal, Oh, Alice, Morency, Louis-Philippe
The utility of vision-based face tracking for dual pointing tasks is evaluated. We first describe a 3-D face tracking technique based on real-time parametric motion-stereo, which is non-invasive,...
Evaluating look-to-talk: A gaze-aware interface in a collaborative environment (2002)
Alice Oh, Harold Fox, Max Van Kleek, Aaron Adler, Krzysztof Gajos, Trevor Darrell
We present “look-to-talk”, a gaze-aware interface for directing a spoken utterance to a software agent in a multiuser collaborative environment. Through a prototype and a Wizard-of-Oz (WOz)...
Face-responsive interfaces: from direct manipulation to perceptive presence (2002)
Trevor Darrell, Konrad Tollmar, Frank Bentley, Neal Checka
Abstract. Systems for tracking faces using computer vision have recently become practical for human-computer interface applications. We are developing prototype systems for face-responsive...
Recovering articulated model topology from observed motion (2002)
Leonid Taycher, Trevor Darrell
Accurate representation of articulated motion is a challenging problem for machine perception. Several successful tracking algorithms have been developed that model human body as an articulated tree....
Evaluating look-to-talk: A gaze-aware interface in a collaborative environment (2002)
Alice Oh, Harold Fox, Max Van Kleek, Aaron Adler, Krzysztof Gajos, Trevor Darrell
We present "look-to-talk", a gaze-aware interface for directing a spoken utterance to a software agent in a multiuser collaborative environment. Through a prototype and a...
Face recognition from long-term observations (2002)
Gregory Shakhnarovich, John W. Fisher, Trevor Darrell
Abstract. We address the problem of face recognition from a large set of images obtained over time- a task arising in many surveillance and authentication applications. A set or a sequence of images...
Evaluating look-to-talk: A gaze-aware interface in a collaborative environment (2002)
Alice Oh, Harold Fox, Max Van Kleek, Aaron Adler, Krzysztof Gajos, Trevor Darrell
We present “look-to-talk”, a gaze-aware interface for directing a spoken utterance to a software agent in a multiuser collaborative environment. Through a prototype and a Wizard-of-Oz (WOz)...
On probabilistic combination of face and gait cues for identification (2002)
Gregory Shakhnarovich, Trevor Darrell
We approach the task of person identification based on face and gait cues. The cues are derived from multiple simultaneous camera views, combined through the visual hull algorithm to create imagery...
Recovering articulated model topology from observed motion (2002)
Leonid Taycher, Trevor Darrell
Tracking human motion is an integral part to developing powerful human-computer interfaces. Several successful tracking algorithms were developed that model human body as an articulated tree. We...
Fast 3D Model Acquisition from Stereo Images (2002)
We propose a fast 3D model acquisition system that aligns intensity and depth images, and reconstructs a textured 3D mesh. 3D views are registered with shape alignment based on intensity gradient...
Evaluating look-to-talk: A gaze-aware interface in a collaborative environment (2002)
Alice Oh, Harold Fox, Max Van Kleek, Aaron Adler, Krzysztof Gajos, Trevor Darrell
We present “look-to-talk”, a gaze-aware interface for directing a spoken utterance to a software agent in a multiuser collaborative environment. Through a prototype and a Wizard-of-Oz (WOz)...
Fast stereo-based head tracking for interactive environments (2002)
Ali Rahimi, Neal Checka, Trevor Darrell
We present a robust implementation of stereo-based head tracking designed for interactive environments with uncontrolled lighting. We integrate fast face detection and drift reduction algorithms with...
3-D Articulated Pose Tracking for Untethered Deictic Reference (2002)
David Demirdjian, Trevor Darrell
Arm and body pose are useful cues for deictic reference-- users naturally extend their arms to objects of interest in a dialog. We present some recent progress on untethered sensing of articulated...
Location estimation with a differential update network (2002)
Given a set of hidden variables with an a-priori Markov structure, we derive an online algorithm which approximately updates the posterior as pairwise measurements between the hidden variables become...
Face recognition from long-term observations (2002)
Gregory Shakhnarovich, John W. Fisher, Trevor Darrell
Abstract. We address the problem of face recognition from a large set of images obtained over time- a task arising in many surveillance and authentication applications. A set or a sequence of images...
Recovering articulated model topology from observed motion (2002)
Leonid Taycher, Trevor Darrell
Accurate representation of articulated motion is a challenging problem for machine perception. Several successful tracking algorithms have been developed that model human body as an articulated tree....
Activity maps for location-aware computing (2002)
David Demirdjian, Konrad Tollmar, Kimberle Koile, Neal Checka, Trevor Darrell
The Problem: Location-based context is important for many applications. Previous systems offered only coarse room-level features or used manually specified room regions to determine fine-scale...
Recovering Articulated Model Topology from Observed Motion (2002)
Leonid Taycher John, Trevor Darrell
Tracking human motion is an integral part to developing powerful human-computer interfaces. Several successful tracking algorithms were developed that model human body as an articulated tree. We...
Range Segmentation Using Visibility Constraints (2001)
Taycher, Leonid, Darrell, Trevor
Visibility constraints can aid the segmentation of foreground objects observed with multiple range images. In our approach, points are defined as foreground if they can be determined to occlude some...
Range Segmentation Using Visibility Constraints (2001)
Taycher, Leonid, Darrell, Trevor
Visibility constraints can aid the segmentation of foreground objects observed with multiple range images. In our approach, points are defined as foreground if they can be determined to occlude some...
Plan-view trajectory estimation with dense stereo background models (2001)
Trevor Darrell, David Demirdjian, Neal Checka, Pedro Felzenszwalb
Proceedings of the International
Audio-visual segmentation and the cocktail party effect (2000)
Trevor Darrell, Paul Viola, Mit Ai Lab
Audio-based interfaces usually suffer when noise or other acoustic sources are present in the environment. For robust audio recognition, a single source must first be isolated. Existing solutions to...
Example based image synthesis of articulated figures (1999)
We present a method for learning complex appearance mappings, such as occur with images of articulated objects. Traditional interpolation networks fail on this case since appearance is not...
Example based image synthesis of articulated figures (1999)
We present a method for learning complex appearance mappings, such as occur with images of articulated objects. Traditional interpolation networks fail on this case since appearance is not...
Pfinder: Real-Time Tracking of the Human Body (1997)
Christopher Wren, Ali Azarbayejani, Trevor Darrell, Alex Pentland
Pfinder is a real-time system for tracking people and interpreting thier behavior. It runs at 10Hz on a standard SGI Indy computer, and has performed reliably on thousands of people in many different...
Modeling, tracking and interactive animation of faces and heads using input from video (1996)
Irfan Essa, Sumit Basu, Trevor Darrell, Alex Pentl
We describe tools that use measurements from video for the extraction of facial modeling and animation parameters, head tracking, and real-time interactive facial animation. These tools share common...
Task-specific Gesture Analysis in Real-Time using Interpolated Views (1996)
Trevor Darrell, Irfan A. Essa, Alex P. Pentl
Hand and face gestures are modeled using an appearance-based approach in which patterns are represented as a vector of similarity scores to a set of view models defined in space and time. These view...
Active Face Tracking and Pose Estimation in an Interactive Room (1996)
Trevor Darrell, Baback Moghaddam, Alex P. Pentland
We demonstrate real-time face tracking and pose estimation in an unconstrained office environment with an active foveated camera. Using vision routines previously implemented for an interactive...
Active Gesture Recognition using Partially Observable Markov Decision Processes (1996)
We present a foveated gesture recognition system that guides an active camera to foveate salient features based on a reinforcement learning paradigm. Using vision routines previously implemented for...
The ALIVE System: Wireless, Full-body Interaction with Autonomous Agents (1996)
Pattie Maes, Trevor Darrell, Bruce Blumberg, Alex Pentland
The cumbersomenature of wired interfaces often limits the range of application of virtual environments. In this paper we discuss the design and implementation of a novel system, called ALIVE, which...
Modeling, Tracking and Interactive Animation of Faces and Heads using Input from Video (1996)
Irfan Essa, Sumit Basu, Trevor Darrell, Alex Pentland
We describe tools that use measurements from video for the extraction of facial modeling and animation parameters, head tracking, and real-time interactive facial animation. These tools share common...
Attention-driven Expression and Gesture Analysis in an Interactive Environment (1995)
Trevor Darrell, Alex P. Pentland
To provide natural user interfaces to interactive environments, accurate and fast recognition of gestures and expressions is needed. We adopt a viewbased gesture recognition strategy that runs in an...
Separation of Transparent Motion into Layers using Velocity-Tuned Mechanisms (1994)
Trevor Darrell, Eero Simoncelli
This paper presents a model for the perception of transparently combined moving images. We advocate a framework consisting of a local motion mechanism which can operate in the presence of...
A Novel Environment for Situated Vision and Behavior (1994)
Trevor Darrell, Pattie Maes, Bruce Blumberg, Alex P. Pentland
We present a new environment for the development of situated vision and behavior algorithms. Our environment allows an unencumbered person to interact with autonomous agents in a simulated graphical...
Irfan A. Essa, Trevor Darrell, Alex Pentland
We describe a computer system that allows real-time tracking of facial expressions. Sparse, fast visual measurements using 2-D templates are used to observe the face of a subject. Rather than track...
Separation of Transparent Motion into Layers using Velocity-Tuned Mechanisms (1994)
Trevor Darrell, Eero Simoncelli
This paper presents a model for the perception of transparently combined moving images. We advocate a framework consisting of a local motion mechanism which can operate in the presence of...
Correlation and Interpolation Networks for Real-time Expression Analysis/Synthesis (1994)
Trevor Darrell, Irfan Essa, Alex Pentland
We describe a framework for real-time tracking of facial expressions that uses neurally-inspired correlation and interpolation methods. A distributed view-based representation is used to characterize...
Evolving Visual Routines (1994)
Michael Patrick Johnson, Pattie Maes, Trevor Darrell
Traditional machine vision assumes that the vision system recovers a a complete, labeled description of the world [ Marr, 1982 ] . Recently, several researchers have criticized this model and...
On the use of "Nulling" Filters to Separate Transparent Motions (1993)
Trevor Darrell, Eero Simoncelli
Transparent motions can be isolated in the Fourier domain, using derivative prefilters that selectively null the spatio-temporal energy consistent with the gradient constraint imposed by each...
Integrated descriptions for vision / (1990)
Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Architecture, 1990.
Sy Bor Wang, David Demirdjian, Trevor Darrell
Detecting communication errors from visual cues during
Second-Order Method for Occlusion Relationships in Motion Layers
A new method is presented to recover the depth ordering of motion layers in a scene. We derive a measure for determining the occlusion relationship between two layers, by testing whether the support...