A Novel Sequence Representation for Unsupervised Analysis of Human Activities (2009)
Raffay Hamid, Siddhartha Maddi, Amos Johnson, Aaron Bobick, Irfan Essa, Charles Isbell
A Non-Invasive Computer Vision System For Reliable Eye Tracking (2009)
Knowing what the user is attending to and what they are looking at is essential for creating attentive user interfaces. Towards this end, we are building a reliable, real-time, non-invasive eye...
Abstract Video Textures (2008)
This paper introduces a new type of medium, called a video texture, which has qualities somewhere between those of a photograph and a video. A video texture provides a continuous infinitely varying...
David Minnen, Charles Isbell, Irfan Essa, Thad Starner
Discovering recurring patterns in time series data is a fundamental problem for temporal data mining. This paper addresses the problem of locating subdimensional motifs in real-valued, multivariate...
web-based interaction. H.5.4 [Information Interfaces and (2008)
Nicholas Diakopoulos, Kurt Luther, Yevgeniy “eugene Medynskiy, Irfan Essa
In an abstract sense, authorship entails the constrained selection or generation of media and the organization and layout of that media in a larger structure. But authorship is more than just...
Abstract Graphcut Textures: Image and Video Synthesis Using Graph Cuts (2008)
Vivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk, Aaron Bobick
This banner was generated by merging the source images in Figure 6 using our interactive texture merging technique. In this paper we introduce a new algorithm for image and video texture synthesis....
Pattern Discovery for Locating Motifs in Multivariate, Real-valued Time-series Data (2008)
David Minnen, Thad Starner, Irfan Essa, Charles Isbell
The problem of locating motifs in multivariate, real-valued time series data concerns the discovery of sets of recurring patterns embedded in the time series. Each set is composed of several...
Pis Umakishore Ramachandran, Kenneth Mackenzie, Steve Deweerth, Irfan Essa, Thad Starner
The proposed research is integrating sensing hardware, embedded processing and distributed system support to build a seamless programming infrastructure for ubiquitous presence applications....
Segmental boosting algorithm for time-seris feature selection (2008)
Pei Yin, Irfan Essa, James M. Rehg
Discriminative feature selection paradigms, e.g., [8, 9] usually consider observation frames in an isolated manner, neglecting temporal dependency in time series. Such temporal relationships provide...
Activity Discovery: Sparse Motifs from Multivariate Time Series (2008)
David Minnen, Thad Starner, Irfan Essa, Charles Isbell
In a set of time series or other sequence data, a motif is a collection of relatively short subsequences that exhibit high self-similarity yet are distinguishable from other subsequences of the data....
Pis Umakishore Ramachandran, Kenneth Mackenzie, Steve Deweerth, Irfan Essa, Thad Starner
The proposed research is integrating sensing hardware, embedded processing and distributed system support to build a seamless programming infrastructure for ubiquitous presence applications....
I.: Videotater: an approach for pen-based digital video segmentation and tagging (2008)
Nicholas Diakopoulos, Irfan Essa
The continuous growth of media databases necessitates development of novel visualization and interaction techniques to support management of these collections. We present Videotater, an experimental...
Hypertext/Hypermedia – theory, user issues. (2008)
Nicholas Diakopoulos, Kurt Luther, Yevgeniy “eugene Medynskiy, Irfan Essa
Authorship entails the constrained selection or generation of media and the organization and layout of that media in a larger structure. But authorship is more than just selection and organization;...
Presenting Movement in a Computer-Based Dance (2008)
Katherine E. Sukel, Richard Catrambone, Irfan Essa, Gabriel Brostow
This article addresses how to present movement information to learners as part of a larger project on developing a nonconventional computational system that teaches ballet. The requirements of such a...
Pis Umakishore Ramachandran, Kenneth Mackenzie, Steve Deweerth, Irfan Essa, Thad Starner
The proposed research is integrating sensing hardware, embedded processing and distributed system support to build a seamless programming infrastructure for ubiquitous presence applications....
Pis Umakishore Ramachandran, Kenneth Mackenzie, Steve Deweerth, Irfan Essa, Thad Starner
The proposed research is integrating sensing hardware, embedded processing and distributed system support to build a seamless programming infrastructure for ubiquitous presence applications....
Pis Umakishore Ramachandran, Kenneth Mackenzie, Steve Deweerth, Irfan Essa, Thad Starner
The proposed research is integrating sensing hardware, embedded processing and distributed system support to build a seamless programming infrastructure for ubiquitous presence applications....
Pis Umakishore Ramachandran, Kenneth Mackenzie, Steve Deweerth, Irfan Essa, Thad Starner
The proposed research is integrating sensing hardware, embedded processing and distributed system support to build a seamless programming infrastructure for ubiquitous presence applications....
Estimating the spatial position of spectral components in audio (2008)
Abstract. One way of separating sources from a single mixture recording is by extracting spectral components and then combining them to form estimates of the sources. The grouping process remains a...
Point projection is the fundamental model for the transformation wrought on by our eye, by cameras, or by numerous other imaging devices. A pinhole camera is a first order approximation used to...
CS7321 Winter 1997: Computer Vision I Notes on Shading (2008)
55> 2 q y 2 , , , E x y , ( ) I x y , ( ) R p q , ( ) -- ( ) 2 l p x 2 p y 2 q x 2 q y 2 + + + ( ) + = l p x y , ( ) p av x y , ( ) T x y p q , , , ( ) p ¶ ¶R + = q x y , ( ) q av x y , ( ) T x...
Augmenting the Capture and Understanding of Everyday Experiences (2008)
Gregory D. Abowd, Christopher Atkeson, Irfan Essa, Blair Macintyre, Elizabeth Mynatt, Colin Potts, ...
s of Individual Research Projects 6 4.1 Investigating research issues in ubiquitous computing: The capture, integration, and access problem . . . . . . . . . . . 6 4.2 Automated understanding of...
Raffay Hamid, Siddhartha Maddi, Aaron Bobick, Irfan Essa
Traditional approaches for activity modeling assume prior knowledge about the structure of activi-ties, based on which explicitly defined models are learned in a supervised manner. However, such...
II. Ubiquitous Smart Spaces II.1 Innovative Capability Envisioned (2007)
Gregory Abowd, Chris Atkeson, Irfan Essa
The next revolutionary advance in smart spaces research is to create and experiment with ubiquitous smart spaces. Current research has created isolated smart rooms and enhanced individuals. This...
Ubiquitous Smart Spaces (2007)
Gregory Abowd, Chris Atkeson, Irfan Essa
s well as including personal mobile components. II.2 Applications and Benefits There is growing interest in using computing technologies to build systems that support our daily lives more...
Visual Coding and Tracking of Speech Related Facial Motions (2007)
We present a new method for video-based coding of facial motions inherent with speaking. We propose a set of four Facial Speech Parameters (FSP): jaw opening, lip rounding, lip closure, and lip...
Spectral Partitioning for Structure from Motion (2007)
Drew Steedly, Irfan Essa, Frank Dellaert
We propose a spectral partitioning approach for large-scale optimization problems, specifically structure from motion. In structure from motion, partitioning methods reduce the problem into smaller...
(School of Mathematics) (2007)
Thomas Carlson, Dr. Greg Turk Adviser, J. Mucha, Dr. Irfan Essa, Dr. James F. O’brien, ...
for my fiancé, Kristi Lee Burns, who thought that staying in school and becoming fake doctors together would be a cool idea iii ACKNOWLEDGEMENTS Much of the material in this dissertation is drawn...
Improving activity discovery with automatic neighborhood estimation (2007)
David Minnen, Thad Starner, Irfan Essa, Charles Isbell
A fundamental problem for artificial intelligence is identifying perceptual primitives from raw sensory signals that are useful for higher-level reasoning. We equate these primitives with initially...
Improving Activity Discovery with Automatic Neighborhood Estimation (2007)
David Minnen Thad, David Minnen, Thad Starner, Irfan Essa, Charles Isbell
A fundamental problem for artificial intelligence is identifying perceptual primitives from raw sensory signals that are useful for higher-level reasoning.
Improving activity discovery with automatic neighborhood estimation (2007)
David Minnen, Thad Starner, Irfan Essa, Charles Isbell
A fundamental problem for artificial intelligence is identifying perceptual primitives from raw sensory signals that are useful for higher-level reasoning. We equate these primitives with initially...
Tree-based classifiers for bilayer video segmentation (2007)
Pei Yin, Antonio Criminisi, John Winn, Irfan Essa
This paper presents an algorithm for the automatic segmentation of monocular videos into foreground and background layers. Correct segmentations are produced even in the presence of large background...
Structure from statistics - unsupervised activity analysis using suffix trees (2007)
Raffay Hamid, Siddhartha Maddi, Aaron Bobick, Irfan Essa
Models of activity structure for unconstrained environments are generally not available a priori. Recent representational approaches to this end are limited by their computational complexity, and...
PROCEDURAL REDUCTION MAPS Approved by: (2007)
Dr. Gregory Turk Adviser, Dr. Irfan Essa, Dr. Jarek Rossignac, Dr. Blair Macintyre
to my wonderful, loving, and beautiful Fianceè who made this possible in all ways-I look forward to seeing You soon iii PREFACE ACM SIGGRAPH is an interesting experience. Everyone has a different...
Discovering multivariate motifs using subsequence density estimation (2007)
David Minnen, Charles L. Isbell, Irfan Essa, Thad Starner
The problem of locating motifs in real-valued, multivariate time series data involves the discovery of sets of recurring patterns embedded in the time series. Each set is composed of several...
Unsupervised analysis of activity sequences using event-motifs (2006)
Raffay Hamid, Siddhartha Maddi, Aaron Bobick, Irfan Essa
We present an unsupervised framework to discover characterizations of everyday human activities, and demonstrate how such representations can be used to extract points of interest in event-streams....
Discovering Characteristic Actions from On-Body Sensor Data (2006)
David Minnen Thad, David Minnen, Thad Starner, Irfan Essa, Charles Isbell
We present an approach to activity discovery, the unsupervised identification and modeling of human actions embedded in a larger sensor stream. Activity discovery can be seen as the inverse of the...
Aware Home: Sensing, Interpretation, and Recognition of Everday Activities (2005)
Presentation given in the Wilby Room of the Georgia Tech Library and Information Center.
A Bayesian View of Boosting and Its Extension (2005)
Bobick, Aaron, Essa, Irfan, Shi, Yifan
In this paper, we provide a Bayesian perspective of boosting framework, which we refer to as Bayesian Integration. Through this perspective, we prove the standard ADABOOST is a special case of the...
Unsupervised Activity Discovery and Characterization from Event-Streams (2005)
Raffay Hamid, Siddhartha Maddi, Amos Johnson, Aaron Bobick, Irfan Essa, Charles Isbell
We present a framework to discover and characterize different classes of everyday activities from event-streams. We begin by representing activities as bags of event n-grams. This allows us to...
Unsupervised Activity Discovery and Characterization From Event-Streams (2005)
Raffay Hamid Siddhartha, Siddhartha Maddi, Amos Johnson, Aaron Bobick, Irfan Essa, Charles Isbell
Introduction: Recognizing what is happening in an environment has many potential applications, ranging from automatic surveillance systems to supporting users in ubiquitous environments. A key step...
Video-based nonphotorealistic and expressive illustration of motion (2005)
We present a semi-automatic approach for adding expressive renderings to images and videos that highlight motions and movement. Our technique relies on motion analysis of video where the motion...
Example-based Rendering of Textural Phenomena (2005)
Vivek Kwatra, Advisors Dr, Aaron Bobick, Dr. Irfan Essa, Vivek Kwatra
Performed research on human body-part labelling and tracking, compression of 2D cel animations, texture synthesis for images and video, and video-based rendering of textural phenomena. 1999-2000...
Texture optimization for example-based synthesis (2005)
Vivek Kwatra, Irfan Essa, Aaron Bobick, Nipun Kwatra
Figure 1: Animating texture using a flow field. Shown are keyframes from texture sequences following a sink. We present a novel technique for texture synthesis using optimization. We define a Markov...
Novel skeletal representation for articulated creatures (2004)
Gabriel J. Brostow, Irfan Essa, Drew Steedly, Vivek Kwatra
Abstract. Volumetric structures are frequently used as shape descriptors for 3D data. The capture of such data is being facilitated by developments in multi-view video and range scanning, extending...
Novel skeletal representation for articulated creatures (2004)
Gabriel J. Brostow, Irfan Essa, Drew Steedly, Vivek Kwatra
Abstract. Volumetric structures are frequently used as shape descriptors for 3D data. The capture of such data is being facilitated by developments in multi-view video and range scanning, extending...
TV Watcher: Distributed Media Analysis and Correlation (2004)
David Hilley Ahmed, Ahmed El-helw, Matthew Wolenetz, Irfan Essa, Phillip Hutto, Umakishore Ramach
The explosion of available content in broadcast media has created a desperate need for applications and prerequisite system architectures to support automatic capture, filtration, categorization,...
Image and video based painterly animation (2004)
“Impressionism”, (bottom row) “Abstract”, “Pointillism”, “Flower ” and “Abstract ” styles. Figure 3 shows some of the brush strokes used. We present techniques for transforming...
Novel skeletal representation for articulated creatures (2004)
Gabriel J. Brostow, Irfan Essa, Drew Steedly, Vivek Kwatra
Abstract. Volumetric structures are frequently used as shape descriptors for 3D data. The capture of such data is being facilitated by developments in multi-view video and range scanning, extending...
Content based image synthesis (2004)
Nicholas Diakopoulos, Irfan Essa, Ramesh Jain
Abstract. A new method allowing for semantically guided image editing and synthesis is introduced. The editing process is made considerably easier and more powerful with our content-aware tool. We...
Propagation networks for recognition of partially ordered sequential action (2004)
Yifan Shi, Yan Huang, David Minnen, Aaron Bobick, Irfan Essa
We present Propagation Networks (P-Nets), a novel approach for representing and recognizing sequential activities that include parallel streams of action. We represent each activity using partially...
Mandatory human participation: A new authentication scheme for building secure systems (2003)
Jun Xu, Richard Lipton, Irfan Essa, Minho Sung, Yong Zhu
Abstract — Mandatory Human Participation (MHP) is a novel authentication scheme that asks the question “are you human?” (instead of “who are you?”), and upon the correct answer to this...
Retargeting Motion Capture Data Using Spacetime Constraints (2003)
James Vanderhyde, Dr. Irfan Essa
Motion capture data can be really handy for creating beautiful and believable animations. However, sometimes the motion that we want is not exactly what was recorded. A new recording session for the...
Expectation Grammars: Leveraging High-Level Expectations for Activity (2003)
Recognition David Minnen, David Minnen, Irfan Essa, Thad Starner
Video-based recognition and prediction of a temporally extended activity can benefit from a detailed description of high-level expectations about the activity. Stochastic grammars allow for an...
ARGMode - Activity Recognition Using Graphical Models (2003)
Raffay Hamid, Yan Huang, Irfan Essa
This paper presents a new framework for tracking and recognizing complex multi-agent activities using probabilistic tracking coupled with graphical models for recognition. We employ statistical...
Graphcut Textures: Image and Video Synthesis Using Graph Cuts (2003)
Vivek Kwatra Arno, Arno Schödl, Irfan Essa, Greg Turk, Aaron Bobick
In this paper we introduce a new algorithm for image and video texture synthesis. In our approach, patch regions from a sample image or video are transformed and copied to the output and then...
Visual coding and tracking of speech related facial motion (2001)
This article present a visual characterization of facial motions inherent with speaking. We propose a set of four Facial Speech Parameters (FSP): jaw opening, lips rounding, lips closure, and lips...
Recognizing Multitasked Activities using Stochastic Context-Free Grammar (2001)
In this paper, we present techniques for characterizing complex, multi-tasked activities that require both exemplars and models. Exemplars are used to represent object context, image features, and...
Propagation of Innovative Information in Non-Linear Least-Squares Structure from Motion (2001)
We present a new technique that improves upon existing structure from motion (SFM) methods. We propose a SFM algorithm that is both recursive and optimal. Our method incorporates innovative...
Visual Coding and Tracking of Speech Related Facial Motion (2001)
We present a new method for video-based coding of facial motions inherent with speaking. We propose a set of four Facial Speech Parameters (FSP): jaw opening, lip rounding, lip closure, and lip...
Visual Coding and Tracking of Speech Related Facial Motion (2001)
We present a new method for video-based coding of facial motions inherent with speaking. We propose a set of four Facial Speech Parameters (FSP): jaw opening, lip rounding, lip closure, and lip...
Machine learning for video-based rendering (2000)
We present techniques for rendering and animation of realistic scenes by analyzing and training on short video sequences. This work extends the new paradigm for computer animation, video textures,...
A Non-invasive Computer Vision System For Reliable Eye Tracking (2000)
Antonio Haro, Irfan Essa, Myron Flickner
Knowing what the user is attending to and what they are looking at is essential for creating attentive user interfaces. Towards this end, we are building a reliable, real-time, non-invasive eye...
Detecting and Tracking Eyes By Using Their Physiological Properties, Dynamics, and Appearance (2000)
Antonio Haro, Myron Flickner, Irfan Essa
Reliable detection and tracking of eyes is an important requirement for attentive user interfaces. In this paper, we present a methodology for detecting eyes robustly in indoor environments in...
Arno Schödl, Richard Szeliski, David H. Salesin, Irfan Essa
This paper introduces a new type of medium, called a video texture, which has qualities somewhere between those of a photograph and a video. A video texture provides a continuous infinitely varying...
Summer Internship Program for Socioeconomically Disadvantaged Undergraduates. (1999)
Overall, the summer 1998 program was a huge success. All the interns were very pleased with the new experiences they were exposed to, while the mentors/advisors were quite pleased at the progress...
Detecting and Tracking Eyes By Using Their Physiological Properties, Dynamics, and Appearance (1999)
Antonio Haro, Myron Flickner, Irfan Essa
Reliable detection and tracking of eyes is an important requirement for attentive user interfaces. In this paper, we present a methodology for detecting eyes robustly in indoor environments in...
Robust Tracking of People by a Mobile Robotic Agent (1999)
Rawesak Tanawongsuwan, Alexander Stoytchev, Er Stoytchev, Irfan Essa
We present methods for tracking people in dynamic and changing environments from camera mounted on a mobile robot. We describe processes to extract color, motion, and depth information from video and...
Robust Tracking of People by a Mobile Robotic Agent (1999)
Rawesak Tanawongsuwan, Er Stoytchev, Irfan Essa
We present methods for tracking people in dynamic and changing environments from camera mounted on a mobile robot. We describe processes to extract color, motion, and depth information from video and...
A System for Tracking and Recognizing Multiple People with Multiple Cameras (1998)
Scott Stillman, Rawesak Tanawongsuwan, Irfan Essa
In this paper we present a robust real-time method for tracking and recognizing multiple people with multiple cameras. Our method uses both static and Pan-Tilt-Zoom (PTZ) cameras to provide visual...
Prosody analysis for speaker affect determination (1997)
Speech is a complex waveform containing verbal (e.g. phoneme, syllable, and word) and nonverbal (e.g. speaker identity, emotional state, and tone) information. Both the verbal and nonverbal aspects...
Facial Expression Recognition Using Image Motion (1997)
Introduction The communicative power of the face makes machine understanding and recognition of human expression an important problem in computer vision. There is a significant amount of research on...
Prosody analysis for speaker affect determination (1997)
www.cc.gatech.edu/gvu
Motion regularization for model-based head tracking (1996)
Sumit Basu, Irfan Essa, Alex Pentl
This paper describes a method for the robust tracking of rigid head motion from video. This method uses a 3D ellipsoidal model of the head and interprets the optical flow in terms of the possible...
Modeling, tracking and interactive animation of faces and heads using input from video (1996)
Irfan Essa, Sumit Basu, Trevor Darrell, Alex Pentl
We describe tools that use measurements from video for the extraction of facial modeling and animation parameters, head tracking, and real-time interactive facial animation. These tools share common...
Motion Regularization for Model-based Head Tracking (1996)
Sumit Basu, Irfan Essa, Alex Pentland
This paper describes a method for the robust tracking of rigid head motion from video. This method uses a 3D ellipsoidal model of the head and interprets the optical flow in terms of the possible...
Modeling, Tracking and Interactive Animation of Faces and Heads using Input from Video (1996)
Irfan Essa, Sumit Basu, Trevor Darrell, Alex Pentland
We describe tools that use measurements from video for the extraction of facial modeling and animation parameters, head tracking, and real-time interactive facial animation. These tools share common...
Facial Expression Recognition using a Dynamic Model and Motion Energy (1995)
Previous efforts at facial expression recognition have been based on the Facial Action Coding System (FACS), a representation developed in order to allow human psychologists to code expression from...
Causal Analysis for Visual Gesture Understanding (1995)
We are exploring the use of high-level knowledge about bodies in the visual understanding of gesture. Our hypothesis is that many gestures are metaphorically derived from the motor programs of our...
A Vision System for Observing and Extracting Facial Action Parameters (1994)
We describe a computer vision system for observing the "action units" of a face using video sequences as input. The visual observation (sensing) is achieved by using an optimal estimation...
Correlation and Interpolation Networks for Real-time Expression Analysis/Synthesis (1994)
Trevor Darrell, Irfan Essa, Alex Pentland
We describe a framework for real-time tracking of facial expressions that uses neurally-inspired correlation and interpolation methods. A distributed view-based representation is used to characterize...
Physically-based Modeling for Graphics and Vision (1993)
Irfan Essa, Stan Sclaroff, Alex Pentland
this paper has appeared as "A Unified Approach for Physical and Geometric Modeling for Graphics and Animation", in Computer Graphics Forum, The International Journal of the Eurographics...