Social Signals, their Function, and Automatic Analysis: (2009)
Ro Vinciarelli, Maja Pantic, Hervé Bourlard, Alex Pentl
Social Signal Processing (SSP) aims at the analysis of social behaviour in both Human-Human and Human-Computer interactions. SSP revolves around automatic sensing and interpretation of social...
Juyang Weng, James Mcclell, Alex Pentl, Olaf Sporns, Ida Stockman, Mriganka Sur, ...
How does one create an intelligent machine? This problem has proven difficult. Over the past several decades, scientists have taken one of three approaches: In the first, which is knowledge-based, an...
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, MANUSCRIPT ID 1 Life Patterns (2009)
Abstract — In this thesis I develop and evaluate computational methods for extracting life’s patterns from wearable sensor data. Life patterns are the reoccurring events in daily behavior, such...
Juyang Weng, James Mcclell, Alex Pentl, Olaf Sporns, Ida Stockman, Mriganka Sur, ...
How does one create an intelligent machine? This problem has proven difficult. Over the past several decades, scientists have taken one of three approaches: In the first, which is knowledge-based, an...
Juyang Weng, James Mcclell, Alex Pentl, Olaf Sporns, Ida Stockman, Mriganka Sur, ...
How does one create an intelligent machine? This problem has proven difficult. Over the past several decades, scientists have taken one of three approaches: In the first, which is knowledge-based, an...
Ali Azarbayejani, Alex Pentl, Brian Curless
[3] H. H. Baker and T. O. Binford. Depth from edge and intensity based stereo. In Proceedings of
Modeling Interactive Agents in ALIVE (2008)
Pattie Maes, Bruce Blumberg, Trevor Darrell, Alex Pentl, Alan Wexelblat
In this video we discuss the design and implementation of a novel system which allows wireless full-body interaction between a human participant and a graphical world inhabited by autonomous agents....
Tactual. Disptays for Wearabte Computing (2008)
Abstract: This paper provides a general overview of tactual displays (i.e., devices that communicate to a user through the sense of touch) and issues concerning the development of such displays for...
Abstract Wearable feedback systems for rehabilitation (2008)
Michael Sung, Carl Marci, Alex Pentl
In this paper we describe LiveNet, a flexible wearable platform intended for long-term ambulatory health monitoring with real-time data streaming and context classification. Based on the MIT Wearable...
Coupled hidden Markov models for complex action recognition (2008)
Matthew Br, Nuria Oliver, Alex Pentl
We present algorithms for coupling and training hidden Markov models (HMMs) to model interacting processes, and demonstrate their superiority to conventional HMMs in a vision task classifying...
Thad Starner, Joshua Weaver, Alex Pentl
We present two real-time hidden Markov model-based systems for recognizing sentence-level continuous American Sign Language (ASL) using a single camera to track the user’s unadorned hands. The...
Social Serendipity: Mobilizing Social (2008)
Software Mobile phones have been adopted faster than any technology in human history and are now available to the majority of people on Earth who earn more than US$5 a day. More than 600 million...
Abstract Automatic Spoken Affect Analysis and Classification (2008)
This paper reports results from early experiments on automatic classification of spoken affect. The task was to classify short spoken sentences into one of two affect classes: approving or...
Machine Understanding of Human Behavior � (2008)
Maja Pantic, Alex Pentl, Anton Nijholt, Thomas Huang
A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this...
Social Signal Processing: State-of-the-art and future perspectives of an emerging domain (2008)
Ro Vinciarelli, Maja Pantic, Hervé Bourlard, Alex Pentl
The ability to understand and manage social signals of a person we are communicating with is the core of social intelligence. Social intelligence is a facet of human intelligence that has been argued...
We present a system for distributed real-time execution of non-rigid physical simulations whose demonstrated efficiency increases nearly linearly as a function of the number of processors. To achieve...
Coupled hidden Markov models for complex action recognition (2007)
Matthew Br, Nuria Oliver, Alex Pentl
We present algorithms for coupling and training hidden Markov models (HMMs) to model interacting processes, and demonstrate their superiority to conventional HMMs in a vision task classifying...
A Consistent Method for Function Approximation in Mesh-based Applications (2007)
Sumit Basu, Kentaro Toyamay, Alex Pentl
Most mesh-based tracking/modeling techniques face a point at which they need to approximate the underlying image function with a smoother, more regular function. This often occurs when gradients need...
Sumit Basu, Kentaro Toyamay, Alex Pentl
Most mesh-based tracking/modeling techniques face apoint at which they need to approximate the underlying image function with a smoother, more regular function. This often occurs when gradients need...
Generalized Ma. tching for Recognition and Retrieval m an Image Database (2007)
Chahab Nastar, Baback Moghaddam, Alex Pentl
We describe a novel approach for image matching based on deformable intensity surfaces. In this approach, the intensity surface of the image is modeled as a deformable 3D mesh in the (x, y, I(x, y))...
Barbara Rosario, Nuria Oliver, Alex Pentl
When building statistical machine learning models from real data one of the most frequently encountered difficulties is the limited amount of training data compared to what is needed by the specific...
Juyang Weng, James Mcclell, Alex Pentl, Olaf Sporns, Ida Stockman, Mriganka Sur, ...
A new synthesis of the neural, behavioral, and computer sciences is on the horizon. The topic that promises to unite these disparate fields is computational autonomous mental development. The term...
Pattie Maes, Trevor Darrell, Bruce Blumberg, Alex Pentl
The cumbersomenature of wired interfaces often limits the range of application of virtual environments. In this paper we discuss the design and implementation of a novel system, called ALIVE, which...
Thad Starner, Jennifer Healey, Dana Kirsch, Rosalind W. Picard, Alex Pentl
Wearable computing moves computation from the desktop to the user. We are forming a community of networked wearable computer users to explore, over a long period, the augmented realities that these...
City of News: cataloguing the World Wide Web through Virtual Architecture (2007)
Flavia Sparacino, Glorianna Davenport, Alex Pentl
“How do we explore the digital box of fragments that pastes together disjunctive arrays of images and sets of data into a seemingly continuous display? ” … We “need to develop new modes of...
Statistical Learning of Physically-Based Deformations (2007)
Chahab Nastar, Baback Moghaddam, Alex Pentl
We describe a novel approach for image matching based on deformable intensity surfaces. In this approach, the intensity surface of the image is modeled as a deformable 3D mesh in the (x; y; I(x;y))...
Social Serendipity: Proximity Sensing and Cueing (2007)
Nathan Eagle And, Nathan Eagle, Alex Pentl
We describe a system that detects wireless devices, queries a centralized server that links the devices with their owners' online profiles, and instigates serendipitous interactions. The system...
to appear in ICCV'99 Probabilistic Object Recognition and Localization (2007)
The appearance of objects consists of regions of local structure as well as dependencies between these regions. The local structure can be characterized by a vector of local features measured by...
Human Computing and Machine Understanding of Human Behavior: A Survey (2006)
Maja Pantic, Alex Pentl, Anton Nijholt, Thomas Huang
A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this...
Human Computing and Machine Understanding of Human Behavior: A Survey (2006)
Maja Pantic, Alex Pentl, Anton Nijholt, Thomas S. Hunag
Abstract. A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground....
Human Computing and Machine Understanding of Human Behavior: A Survey (2006)
Maja Pantic, Alex Pentl, Anton Nijholt, Thomas Huang
A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this...
Abstract Human dynamics: computation for organizations (2004)
Alex Pentl, Tanzeem Choudhury, Nathan Eagle, Push Singh
The human dynamics group at the MIT Media Laboratory proposes that active pattern analysis of face-to-face interactions within the workplace can radically improve the functioning of the organization....
Characterizing Social Networks using the Sociometer (2004)
Intel Research Seattle, Alex Pentland, Tanzeem Choudhury, Tanzeem Choudhury, Tanzeem Choudhury, Alex Pentl
Knowledge of how groups of people interact is important in many disciplines, e.g. organizational behavior, social network analysis, knowledge management and ubiquitous computing. Existing studies of...
Learning Communities: Connectivity and Dynamics of Interacting Agents (2003)
Tanzeem Choudhury, Brian Clarkson, Sumit Basu, Alex Pentl
this paper, 20 iterations were sufficient to ensure convergence
Learning words from sights and sounds: A computational model (2002)
This paper presents a model of word acquisition which learns from multimodal sensory input. Set in an information theoretic framework, the model acquires a lexicon by finding and statistically...
Statistical Imitative Learning from Perceptual Data (2002)
Imitative learning has recently piqued the interest of various fields including neuroscience, cognitive science and robotics. In computational behavior modeling and development, it promises an...
Tanzeem Choudhury, James M. Rehg, Vladimir Pavlović, Alex Pentl
Bayesian networks are an attractive modeling tool for human sensing, as they combine an intuitive graphical representation with ef£cient algorithms for inference and learning. Earlier work has...
Towards measuring human interactions in conversational settings (2001)
Sumit Basu, Tanzeem Choudhury, Brian Clarkson, Alex Pentl
The Facilitator Room is a conference room that is outfitted with sensors and actuators in order to observe and influence human behavior in conversational settings. This work describes our efforts...
Situation aware computing with wearable computers (2000)
Bernt Schiele, Thad Starner, Brad Rhodes, Brian Clarkson, Alex Pentl
1 Motivation for contextual aware computing For most computer systems, even virtual reality systems, sensing techniques are a means of getting input directly from the user. However, wearable sensors...
Tracking conversational context for machine mediation of human discourse (2000)
Tony Jebara, Yuri Ivanov, Ali Rahimi, Alex Pentl
We describe a system that tracks conversational context using speech recognition and topic modeling. Topics are described by computing the frequency of words for each class. We thus reliably detect,...
Wearable Phased Arrays for Sound Localization and Enhancement (2000)
Sumit Basu, Steve Schwartz, Alex Pentl
We present the idea of a flexible phased array of microphones for wearable computers. We show how such an array can be used for both source localization and signal enhancement. As a result, it can...
Flavia Sparacino, Glorianna Davenport, Alex Pentl
Wearable computing provides a means to transform the architecture and the space surrounding us into a memory device and a storytelling agent. We assembled a wearable computer specifically aimed to...
Lafter: a real-time face and lips tracker with facial expression recognition (2000)
This paper describes an active-camera real-time system for tracking, shape description, and classification of the human face and mouth expressions using only a PC or equivalent computer. The system...
Tracking conversational context for machine mediation of human discourse (2000)
Tony Jebara, Yuri Ivanov, Ali Rahimi, Alex Pentl
We describe a system that tracks conversational context using speech recognition and topic modeling. Topics are described by computing the frequency of words for each class. We thus reliably detect,...
Tracking Conversational Context for Machine Mediation of Human Discourse (2000)
Tony Jebara, Yuri Ivanov, Ali Rahimi, Alex Pentl
We describe a system that tracks conversational context using speech recognition and topic modeling. Topics are described by computing the frequency of words for each class. We thus reliably detect,...
Tracking conversational context for machine mediation of human discourse (2000)
Tony Jebara, Yuri Ivanov, Ali Rahimi, Alex Pentl
We describe a system that tracks conversational context using speech recognition and topic modeling. Topics are described by computing the frequency of words for each class. We thus reliably detect,...
Personalizing Smart Environments: Face Recognition for Human Interaction (2000)
A smart environment is one that is able to identify people, interpret their actions, and react appropriately. Thus, one of the most important building blocks of smart environments is a person identi...
Multimodal person recognition using unconstrained audio and video (1999)
Tanzeem Choudhury, Brian Clarkson, Tony Jebara, Alex Pentl
We propose a person identi cation technique that can recognize and verify people from unconstrained video and audio. We do not expect fully frontal face image or clean speech as our input. Our...
Multimodal person recognition using unconstrained audio and video (1999)
Tanzeem Choudhury, Brian Clarkson, Tony Jebara, Alex Pentl
We propose a person identi cation technique that can recognize and verify people from unconstrained video and audio. We do not expect fully frontal face image or clean speech as our input. Our...
An Interactive Computer Vision System DyPERS: Dynamic Personal Enhanced Reality System (1999)
Bernt Schiele Nuria, Nuria Oliver, Tony Jebara, Alex Pentl
. DyPERS, 'Dynamic Personal Enhanced Reality System', uses augmented reality and computer vision to autonomously retrieve 'media memories' based on associations with real objects...
Bayesian modeling of facial similarity (1998)
Baback Moghaddam, Tony Jebara, Alex Pentl
In previous work [6, 9, 10], we advanced a new technique for direct visual matching of images for the purposes of face recognition and image retrieval, using a probabilistic measure of similarity...
Visual context awareness via wearable computing (1998)
Thad Starner, Bernt Schiele, Alex Pentl
Small, body-mounted video cameras enable a di erent style of computing interface. As processing power increases, a computer can spend more time observing its user and modeling his or her context to...
Real-time american sign language recognition from video using hidden Markov models (1998)
Thad Starner, Student Member, Joshua Weaver, Alex Pentl, Ieee Computer Society
Abstract—We present two real-time hidden Markov model-based systems for recognizing sentence-level continuous American Sign Language (ASL) using a single camera to track the user’s unadorned...
3D modeling and tracking of human lip motions (1998)
Sumit Basu, Nuria Oliver, Alex Pentl
We address the problem of tracking and reconstructing 3D human lip motions from a 2D view. This problem is challenging due both to the complex nature of lip motions and the minimal data available...
Graphical models for recognizing human interactions (1998)
Nuria M. Oliver, Barbara Rosario, Alex Pentl
We describe a real-time computer vision and machine learning system for modeling and recognizing human actions and interactions. Two different domains are explored: recognition of two-handed motions...
Mixtures of eigenfeatures for real-time structure from texture (1998)
Tony Jebara, Kenneth Russell, Alex Pentl
We describe afacemodeling system which estimates complete facial structure and texture from a real-time video stream. The system begins with a face tracking algorithm which detects and stabilizes...
Maximum conditional likelihood via bound maximization and the CEM algorithm (1998)
We present the CEM (Conditional Expectation Maximization) algorithm as an extension of the EM (Expectation Maximization) algorithm to conditional density estimation under missing data. A bounding and...
Graphical models for recognizing human interactions (1998)
Nuria M. Oliver, Barbara Rosario, Alex Pentl
We describe a real-time computer vision and machine learning system for modeling and recognizing human actions and interactions. Two different domains are explored: recognition of two-handed motions...
Auditory Context Awareness via Wearable Computing (1998)
Brian Clarkson, Nitin Sawhney, Alex Pentl
We describe a system for obtaining environmental context through audio for applications and user interfaces. We are interested in identifying specific auditory events such as speakers, cars, and...
Maximum Conditional Likelihood via Bound Maximization and the CEM Algorithm (1998)
Tony Jebara And, Tony Jebara, Alex Pentl
We present the CEM (Conditional Expectation Maximization) algorithm as an extension of the EM (Expectation Maximization) algorithm to conditional density estimation under missing data. A bounding and...
MIT Media Lab 20 Ames Street, Rm. E15-388 Cambridge, MA 01239 (dkroy,sandy)@media.mit.edu (1998)
Ia Mit Edu, Deb Roy, Alex Pentl
We are creating human machine interfaces which let people communicate with machines using natural modalities including speech and gesture. A problem with currentmultimodal interfaces is that users...
Maximum Conditional Likelihood via Bound Maximization and the CEM Algorithm (1998)
We present the CEM (Conditional Expectation Maximization) algorithm as an extension of the EM (Expectation Maximization) algorithm to conditional density estimation under missing data. A bounding and...
Auditory Context Awareness via Wearable Computing (1998)
Brian Clarkson, Nitin Sawhney, Alex Pentl
We describe a system for obtaining environmental context through audio for applications and user interfaces. We are interested in identifying specific auditory events such as speakers, cars, and...
Multimodal Person Recognition using Unconstrained Audio and Video (1998)
Tanzeem Choudhury, Brian Clarkson, Tony Jebara, Alex Pentl
We propose a person identification technique that can recognize and verify people from unconstrained video and audio. We do not expect fully frontal face image or clean speech as our input. Our...
Multimodal Person Recognition using Unconstrained Audio and Video (1998)
Tanzeem Choudhury, Brian Clarkson, Tony Jebara, Alex Pentl
We propose a person identification technique that can recognize and verify people from unconstrained video and audio. We do not expect fully frontal face image or clean speech as our input. Our...
Maximum Conditional Likelihood via Bound Maximization and the CEM Algorithm (1998)
We present the CEM (Conditional Expectation Maximization) algorithm as an extension of the EM (Expectation Maximization) algorithm to conditional density estimation under missing data. A bounding and...
Bayesian modeling of facial similarity (1998)
Baback Moghaddam, Tony Jebara, Alex Pentl
In previous work [6, 9,10],weadvanced a new technique for direct visual matching of images for the purposes of face recognition and image retrieval, using a probabilistic measure of similarity based...
We propose a novel technique for direct visual matching of images for the purposes of face recognition and database search. Specifically, we argue in favor of a probabilistic measure of similarity,...
Eric Goubault. Static analyses of operations (1998)
Thad Starner, Bernt Schiele, Bradley J. Rhodes, Tony Jebara, Nuria Oliver, Joshua Weaver, ...
Besides the obvious advantage of mobility, wearable computing o ers intimacy with the user for augmented realities. A model of the user is as important as a model of the physical world for creating a...
Parametrized structure from motion for 3d adaptive feedback tracking of faces (1997)
A real-time system is described for automatically detecting, modeling and tracking faces in 3D. A closed loop approach is proposed which utilizes structure from motion to generate a 3D model of a...
Tony Jebara, Cyrus Eyster, Josh Weaver, Thad Starner, Alex Pentl
We propose a practical application of wearable computing and augmented reality which enhances the game of billiards. A vision algorithm is implemented which operates in interactive-time with the user...
A wearable computer based american sign language recognizer (1997)
Thad Starner, Joshua Weaver, Alex Pentl
Modern wearable computer designs package workstation level performance in systems small enough to be worn as clothing. These machines enable technology to be brought where it is needed the most for...
Tactual displays for wearable computing (1997)
This paper provides a general overview of tactual displays (i.e., devices that communicate to a user through the sense of touch) and issues concerning the development of such displays for wearable...
A three-dimensional model of human lip motions trained from video (1997)
We present a 3D model of human lips and develop a framework for training it from real data. The model starts o with generic physics speci ed with the nite element method and \learns " the...
Tony Jebara, Cyrus Eyster, Josh Weaver, Thad Starner, Alex Pentl
We propose a practical application of wearable computing and augmented reality which enhances the game of billiards. A vision algorithm is implemented which operates in interactive-time with the user...
Tony Jebara, Cyrus Eyster, Josh Weaver, Thad Starner, Alex Pentl
We propose a practical application of wearable computing and augmented reality which enhances the game of billiards. A vision algorithm is implemented which operates in interactive-time with the user...
Flavia Sparacino, Glorianna Davenport, Alex Pentl
“How do we explore the digital box of fragments that pastes together disjunctive arrays of images and sets of data into a seemingly continuous display? ” … We “need to develop new modes of...
Wearable Audio Computing: A Survey of Interaction Techniques (1997)
Deb Roy Nitin, Deb Roy, Nitin Sawhney, Chris Schm, Alex Pentl
We consider wearable computing applications which rely on audio as a primary medium of the interface. This paper surveys a range of interaction techniques which may be applied to the design of...
MIT Media Laboratory, Perceptual Computing Technical Report 401 (1997)
Submitted To Cvpr, Tony S. Jebara, Alex Pentl
A real-time system is described for automatically detecting, modeling and tracking faces in 3D. A closed loop approach is proposed which utilizes structure from motion to generate a 3D model of a...
Augmented reality through wearable computing (1997)
Thad Starner, Steve Mann, Bradley Rhodes, Jeffrey Levine, Jennifer Healey, Dana Kirsch, ...
Wearable computing moves computation from the desktop to the user. We are forming a community of networked wearable computer users to explore, over a long period, the augmented realities that these...
Motion regularization for model-based head tracking (1996)
Sumit Basu, Irfan Essa, Alex Pentl
This paper describes a method for the robust tracking of rigid head motion from video. This method uses a 3D ellipsoidal model of the head and interprets the optical flow in terms of the possible...
Vision-Steered Audio for Interactive Environments (1996)
Sumit Basu, William Gardner, Ali Azarbayejani, Alex Pentl
We present novel techniques for obtaining and producing audio information in an interactive virtual environment using vision information. These techniques are free of mechanisms that would encumber...
Modeling, tracking and interactive animation of faces and heads using input from video (1996)
Irfan Essa, Sumit Basu, Trevor Darrell, Alex Pentl
We describe tools that use measurements from video for the extraction of facial modeling and animation parameters, head tracking, and real-time interactive facial animation. These tools share common...
Bayesian face recognition using deformable intensity surfaces (1996)
Baback Moghaddam, Chahab Nastar, Alex Pentl
We describe a novel technique for face recognition based on deformable intensity surfaces which incorporates both the shape and texture components of the 2D image. The intensity surface of the facial...
A Bayesian similarity measure for direct image matching (1996)
Baback Moghaddam, Chahab Nastar, Alex Pentl
We propose a probabilistic similarity measure for direct image matching based on a Bayesian analysis of image deformations. We model two classes of variation in object appearance: intra-object and...
Automatic Spoken Affect Analysis and Classification (1996)
Deb Roy And, Deb Roy, Alex Pentl
This paper reports results from early experiments on automatic classification of spoken affect. The task was to classify short spoken sentences into one of two affect classes: approving or...
Real-Time 3-D Tracking of the Human Body (1996)
Ali Azarbayejani, Christopher Wren, And Alex Pentland, Alex Pentl
People are the central element in the whole enterprise of multimedia and communications and thus visual interpretation of humans and their movements is an important problem for computers. Here we...
Chloe M. Chao, Flavia Sparacino, Alex Pentl, Joe Marks, Chloe M. Chao, Alex Pentland, ...
Full-body, unencumbered, gestural interfaces offer new possibilities for computer painting tools. However, traditional painting metaphors and current gestural input technology are not well suited to...
Bayesian face recognition using deformable intensity surfaces (1996)
Baback Moghaddam, Chahab Nastar, Alex Pentl
We describe a novel technique for face recognition based on deformable intensity surfaces which incorporates both the shape and texture components of the 2D image. The intensity surface of the facial...
Wearable computing and augmented reality (1995)
Thad Starner, Steve Mann, Bradley Rhodes, Jennifer Healey, Kenneth B. Russell, Jeffrey Levine, ...
ABSTRACT: Wearable computing will change the current paradigms of human-computer interaction. With heads-up displays, unobtrusive input devices, personal wireless local area networks, and a host of...
Hyperplex: a world of 3d interactive digital movies (1995)
Flavia Sparacino, Christopher Wren, Alex Pentl, Glorianna Davenport
We present a new environment for browsing a visual landscape inhabited by digital movies that live, interact and play in a graphical virtual world. The movies are modeled as autonomous agents which...
An automatic system for model-based coding of faces (1995)
We present a fully automatic system for 2D model-based image coding of human faces for potential applications such as video telephony, database image compression, and face recognition. The system...
Shape analysis of brain structures using physical and experimental modes (1994)
John Martin, Alex Pentl, Ron Kikinis
This paper presents a framework for analyzing the shape deformation of structures within the human brain. A mathematical model is developed describing
A Vision System for Observing and Extracting Facial Action Parameters (1994)
We describe a computer vision system for observing the "action units" of a face using video sequences as input. The visual observation (sensing) is achieved by using an optimal estimation...
Recursive Estimation for CAD Model Recovery (1994)
Ali Azarbayejani, Tinsley Galyean, Bradley Horowitz, Alex Pentl
We describe a system for semiautomatically extracting 3-D object models from raw, uncalibrated video. The system utilizes a recursive estimator to accurately recover camera motion, pointwise...
Modal Matching for Correspondance and Recognition (1993)
We describe a new method for establishing correspondence, computing canonical descriptions, and recognizing objects that is based on the idea of describing objects by their generalized symmetries, as...
D. D. Hoffman, Whitman Richards, Alex Pentl, John Rubin, Joseph Scheuhammer
Abstract. A complete theory of object recognition is an impossibility m not simply. because of the multiplicity of visual cues we exploit in elegant coordination to identify an object, but primarily...