Geometry-driven photorealistic facial expression synthesis (2009)
Qingshan Zhang, Zicheng Liu, Senior Member, Baining Guo, Senior Member, Demetri Terzopoulos, ...
Abstract—Expression mapping (also called performance driven animation) has been a popular method for generating facial animations. A shortcoming of this method is that it does not generate...
Chinese Academy of Sciences (2009)
Xiubo Geng, Andrew Arnold, Tie-yan Liu, Hang Li, Tao Qin, Heung-yeung Shum
Many ranking models have been proposed in information retrieval, and recently machine learning techniques have also been applied to ranking model construction. Most of the existing methods do not...
Abstract Direct Manipulation of Subdivision Surfaces on GPUs (2008)
Kun Zhou, Xin Huang, Weiwei Xu, Baining Guo, Heung-yeung Shum
We present an algorithm for interactive deformation of subdivision surfaces, including displaced subdivision surfaces and subdivision surfaces with geometric textures. Our system lets the user...
Jiang Li, Kun Zhou, Yong Wang, Heung-yeung Shum
This paper introduces a novel image-based rendering system to capture, represent and render real world and synthetic scenes. In our system, a longitudinally aligned camera array is mounted on a...
An efficient approach to learning inhomogeneous Gibbs model (2008)
Ziqiang Liu, Hong Chen, Heung-yeung Shum
Inhomogeneous Gibbs model (IGM) [4] is an effective maximum entropy model in characterizing complex highdimensional distributions. However, its training process is so slow that the applicability of...
Abstract TextureMontage: Seamless Texturing of Arbitrary Surfaces From Multiple Images (2008)
Kun Zhou, Xi Wang, Yiying Tong, Mathieu Desbrun, Baining Guo, Heung-yeung Shum
We propose a technique, called TextureMontage, to seamlessly map a patchwork of texture images onto an arbitrary 3D model. A texture atlas can be created through the specification of a set of...
To appear in the ACM SIGGRAPH conference proceedings Poisson Matting (2008)
Jian Sun, Jiaya Jia, Chi-keung Tang, Heung-yeung Shum
Figure 1: Pulling of matte from a complex scene. From left to right: a complex natural image for existing matting techniques where the color background is complex, a high quality matte generated by...
Stereo Reconstruction from (2008)
Multiperspective Panoramas, Yin Li, Heung-yeung Shum, Senior Member, Chi-keung Tang, Ieee Computer Society, ...
Abstract—A new approach to computing a panoramic (360 degrees) depth map is presented in this paper. Our approach uses a large collection of images taken by a camera whose motion has been...
Tai-pang Wu, Chi-keung Tang, Michael S. Brown, Heung-yeung Shum
Figure 1: An example of 3D markup using Shape Palettes. The ‘question mug ’ is created by linking 2D primitives drawn in the freehand view to their corresponding 3D shape on the “shape palette...
Eurographics Symposium on Rendering (2005) (2008)
Kavita Bala, Philip Dutré (editors, Multiresolution Reflectance Filtering, Ping Tan, Stephen Lin, Long Quan, ...
Physically-based reflectance models typically represent light scattering as a function of surface geometry at the pixel level. With changes in viewing resolution, the geometry imaged within a pixel...
Abstract Direct Manipulation of Subdivision Surfaces on GPUs (2008)
Kun Zhou, Xin Huang, Weiwei Xu, Baining Guo, Heung-yeung Shum
We present an algorithm for interactive deformation of subdivision surfaces, including displaced subdivision surfaces and subdivision surfaces with geometric textures. Our system lets the user...
Image Deblurring with Blurred/Noisy Image Pairs (2008)
Lu Yuan, Jian Sun, Long Quan, Heung-yeung Shum
(with shutter speed of 1/100 second, and ISO 1600) due to insufficient light. (c) Noisy image enhanced by adjusting level and gamma. (d) Our deblurred image. Abstract Taking satisfactory photos under...
Generic slow-motion replay detection in sports video (2008)
Lei Wang, Xu Liu, Steve Lin, Guangyou Xu, Heung-yeung Shum
Slow-motion replays highlight important and exciting events in sports videos. Previous works for slow-motion replay detection, however, are usually limited to only some specific production rules such...
Interactive Offline Tracking for Color Objects (2008)
Yichen Wei, Jian Sun, Xiaoou Tang, Heung-yeung Shum
In this paper, we present an interactive offline tracking system for generic color objects. The system achieves 60-100 fps on a 320 × 240 video. The user can therefore easily refine the tracking...
The Visual Computer manuscript No. (will be inserted by the editor) (2008)
Ying Song, Yanyun Chen, Xin Tong, Stephen Lin, Jiaoying Shi, Baining Guo, ...
Abstract The appearance of an inhomogeneous translucent material depends substantially on its volumetric variations and their effects upon subsurface scattering. For efficient rendering that accounts...
Abstract Precomputed Shadow Fields for Dynamic Scenes (2008)
Kun Zhou, Yaohua Hu, Stephen Lin, Baining Guo, Heung-yeung Shum
We present a soft shadow technique for dynamic scenes with moving objects under the combined illumination of moving local light sources and dynamic environment maps. The main idea of our technique is...
c ○ World Scientific Publishing Company ARTISTIC IMAGE GENERATION BY DEVIATION MAPPING (2008)
Liu Wenyin, Xin Tong, Xu Yingqing, Heung-yeung Shum, Zhong Hua
This paper describes a novel technique (deviation mapping) that generates images with artistic appearances, e.g. spray painted wall, embossment, cashmere painting, etc. Our technique employs a...
Jiang Li, Heung-yeung Shum, Qunsheng Peng
Traditional ray tracing methods are based on Newton’s classical mechanics. They cannot reproduce the visual phenomena of high-speed motion. In this paper, we develop an improved spacetime ray...
Abstract TextureMontage: Seamless Texturing of Arbitrary Surfaces From Multiple Images (2008)
Kun Zhou, Xi Wang, Yiying Tong, Mathieu Desbrun, Baining Guo, Heung-yeung Shum
We propose a technique, called TextureMontage, to seamlessly map a patchwork of texture images onto an arbitrary 3D model. A texture atlas can be created through the specification of a set of...
Abstract Subspace Gradient Domain Mesh Deformation (2008)
Jin Huang, Xiaohan Shi, Xinguo Liu, Kun Zhou, Li-yi Wei, Shang-hua Teng, ...
In this paper we present a general framework for performing constrained mesh deformation tasks with gradient domain techniques. We present a gradient domain technique that works well with a wide...
Rendering Handwritten Characters (2008)
Sara Su, Chenyu Wu, Ying-qing Xu, Heung-yeung Shum
Introduction: As users of pen-based computers rely increasingly on freehand interaction, they become ever more sensitive to the on-screen appearance of their digital handwriting. We describe a method...
Capturing and rendering geometry details for BTFmapped surfaces (2008)
Jiaping Wang, Jiaping Wang, Xin Tong, Xin Tong, Yanyun Chen, Yanyun Chen, ...
Abstract Bidirectional texture functions or BTFs accurately model reflectance variation at a fine (meso-) scale as a function of lighting and viewing direction. BTFs also capture view-dependent...
Correction to Construction of Panoramic Image Mosaics with Global and Local Alignment ∗ (2008)
Heung-yeung Shum, Richard Szeliski
2000), we introduce the following optimization, which underlies the efficiency of our algorithm: An important element of this framework, which we exploit, is to perform the motion estimation between...
Abstract Color Sketch Generation (2008)
Fang Wen, Qing Luan, Lin Liang, Ying-qing Xu, Heung-yeung Shum
In this paper, we introduce an NPR system to generate color sketches in a free-hand drawing style. From observation, we found out that in a free-hand style both shape and color of image regions are...
PicToon: A Personalized Image-based Cartoon System (2008)
Hong Chen, Nan-ning Zheng, Lin Liang, Yan Li, Ying-qing Xu, Heung-yeung Shum
In this paper, we present PicToon, a cartoon system which can generate a personalized cartoon face from an input Picture. PicToon is easy to use and requires little user interaction. Our system...
Abstract Object Representation and Rendering using Inverse Concentric Mosaics (2008)
Lifeng Wang, Heung-yeung Shum, Sing Bing Kang
Three of the most basic tasks in computer vision and graphics are capturing, representing, and rendering objects or scenes in as photorealistic a manner as possible. In this paper, we describe our...
PicToon: A Personalized Image-based Cartoon System (2008)
Hong Chen, Lin Liang, Yan Li, Ying-qing Xu, Heung-yeung Shum
In this paper, we present PicToon, a cartoon system which can generate a personalized cartoon face from an input image. PicToon is easy to use and requires little user interaction. Our system...
Image Vectorization using Optimized Gradient Meshes ACM Reference Format (2008)
Jian Sun, Lin Liang, Fang Wen, Heung-yeung Shum
Copyright Notice Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profi...
Eurographics Symposium on Rendering (2005) (2008)
Kavita Bala, Philip Dutré (editors, Multiresolution Reflectance Filtering, Ping Tan, Stephen Lin, Long Quan, ...
Physically-based reflectance models typically represent light scattering as a function of surface geometry at the pixel level. With changes in viewing resolution, the geometry imaged within a pixel...
Lifeng Wang, Stephen Lin, Seungyong Lee, Baining Guo, Heung-yeung Shum, Senior Member
Abstract — We present a 2D feature-based technique for morphing 3D objects represented by light fields. Existing light field morphing methods require the user to specify corresponding 3D feature...
Physically Based Real-time Animation of Hangings (2008)
Chiyi Cheng, Ying-qing Xu, Jiaoying Shi, Heung-yeung Shum
In this paper, we propose a new physically based model for the real-time animation of hangings, which not only greatly speeds up the rendering, but also maintains visually appealing results. Our...
Sara L. Su, Chenyu Wu, Heung-yeung Shum
With the growing popularity of pen-based computers comes the need to display clear handwritten characters at small sizes on low-resolution displays. This paper describes a method for automatically...
All-Frequency Precomputed Radiance Transfer for Glossy Objects (2008)
H. W. Jensen, A. Keller (editors, Xinguo Liu, Peter-pike Sloan, Heung-yeung Shum, John Snyder
We introduce a method based on precomputed radiance transfer (PRT) that allows interactive rendering of glossy surfaces and includes shadowing effects from dynamic, “all-frequency ” lighting....
Accepted by International Journal of Computer Vision Face Hallucination: Theory and Practice (2008)
Ce Liu, Heung-yeung Shum, William T. Freeman
In this paper, we study face hallucination, or synthesizing a high-resolution face image from an input low-resolution image, with the help of a large collection of other high-resolution face images....
Stan Z. Li, Heung-yeung Shum, Zhenqiu Zhang, Hongjiang Zhang
AdaBoost [3] minimizes an upper error bound which is an exponential function of the margin on the training set [14]. However, the ultimate goal in applications of pattern classification is always...
Yanxi Liu and James Hays Carnegie Mellon University Digital Papercutting (2008)
Ying-qing Xu, Heung-yeung Shum
Figure 1: Given a papercut-pattern (a), our algorithm performs analysis, fold-then-cut plan generation and synthesis. The algorithm identifies the dihedral symmetry group D2x4 in the outer-ring of...
Secure Human-Computer Identification against Peeping Attacks (SecHCI): A Survey (2007)
It is an interesting problem how a human can prove its identity to a trustworthy (local or remote) computer with untrustworthy input devices and via an insecure channel controlled by adversaries. Any...
Stylizing Motion with Drawings (2007)
D. Breen, M. Lin (editors, Yin Li, Michael Gleicher, Ying-qing Xu, Heung-yeung Shum
In this paper, we provide a method that injects the expressive shape deformations common in traditional 2D animation into an otherwise rigid 3D motion captured animation. We allow a traditional...
IMAGE-BASED WALKTHROUGH OVER THE INTERNET (2007)
Jiang Li, Yiying Tong, Yong Wang, Heung-yeung Shum, Ya-qin Zhang
In this paper, we describe a client-server image-based walkthrough system. This system employs pictures, panoramas, and concentric mosaics captured from real scenes. These data are compressed and...
Zhuowen Tu, Song-chun Zhu, Heung-yeung Shum
for image segmentation in the Bayesian statistical framework. The paper contributes to image segmentation in three aspects. Firstly, it designs eective and well balanced Markov Chain dynamics to...
technique called Inverse Concentric Mosaics (2007)
Lifeng Wang, Heung-yeung Shum, Sing Bing Kang
In this paper, we describe an image-based rendering
1 Artificial Intelligence and Robotics Lab, (2007)
Tian-shu Wang, Heung-yeung Shum, Ying-qing Xu, Nan-ning Zheng
Abstract. Recognition of human gestures is important for analysis and indexing of video. To recognize human gestures on video, generally a large number of training examples for each individual...
ShapePalettes : interactive normal transfer via sketching (2007)
Wu, Tai-Pang, Tang, Chi-Keung, Brown, Michael S., Shum, Heung-Yeung
We present a simple interactive approach to specify 3D shape in a single view using “shape palettes”. The interaction is as follows: draw a simple 2D primitive in the 2D view and then specify its...
Face poser: interactive modeling of 3D facial expressions using model priors (2007)
Manfred Lau, Jinxiang Chai, Ying-qing Xu, Heung-yeung Shum
In this paper, we present an intuitive interface for interactively posing 3D facial expressions. The user can create and edit facial expressions by drawing freeform strokes, or by directly dragging...
Blurred/non-blurred image alignment using sparseness prior (2007)
Lu Yuan, Jian Sun, Long Quan, Heung-yeung Shum
Abstract Aligning a pair of blurred and non-blurred images is a prerequisite for many image and video restoration and graphics applications. The traditional alignment methods such as direct and...
Learning to detect a salient object (2007)
Tie Liu, Jian Sun, Nan-ning Zheng, Xiaoou Tang, Heung-yeung Shum
Abstract We study visual attention by detecting a salient object in an input image. We formulate salient object detection as an image segmentation problem, where we separate the salient object from...
Classification via minimum incremental coding length (MICL (2007)
John Wright, Yi Ma, Yangyu Tao, Zhouchen Lin, Heung-yeung Shum
We present a simple new criterion for classification, based on principles from lossy data compression. The criterion assigns a test sample to the class that uses the minimum number of additional bits...
Principal Component Analysis with Missing Data and its Application to Object Modeling (2006)
Shum, Heung-yeung, Ikeuchi, Katsushi, Reddy, Raj
Observation-based object modeling often requires integration of shape descriptions from different views. In current conventional methods, to sequentially merge multiple views, accurate description of...
Single-view relighting with normal map painting (2006)
Makoto Okabe, Gang Zeng, Yasuyuki Matsushita, Takeo Igarashi, Long Quan, Heung-yeung Shum
This paper presents an interactive technique for changing the illumination of objects in a real photograph. 1 Because lighting effects are very sensitive to surface normal perturbations but hard to...
Jingdong Wang, Jian Sun, Long Quan, Xiaoou Tang, Heung-yeung Shum
In this paper, we address a novel problem of automatically creating a picture collage from a group of images. Picture collage is a kind of visual image summary — to arrange all input images on a...
Accurate face alignment using shape constrained Markov network (2006)
Lin Liang, Fang Wen, Ying-qing Xu, Xiaoou Tang, Heung-yeung Shum
In this paper, we present a shape constrained Markov network for accurate face alignment. The global face shape is defined as a set of weighted shape samples which are integrated into the Markov...
Accurate face alignment using shape constrained Markov network (2006)
Lin Liang, Fang Wen, Ying-qing Xu, Xiaoou Tang, Heung-yeung Shum
In this paper, we present a shape constrained Markov network for accurate face alignment. The global face shape is defined as a set of weighted shape samples which are integrated into the Markov...
Jingdong Wang, Jian Sun, Long Quan, Xiaoou Tang, Heung-yeung Shum
In this paper, we address a novel problem of automatically creating a picture collage from a group of images. Picture collage is a kind of visual image summary — to arrange all input images on a...
Illumination Brush: Interactive Design of Image-based Lighting (2006)
Makoto Okabe, Yasuyuki Matsushita, Takeo Igarashi, Heung-yeung Shum
To help artists design customized lighting environments, we present a simple user interface for designing image-based lighting. Our system allows the artist to directly specify the appearance of the...
Jian Sun, Weiwei Zhang, Xiaoou Tang, Heung-yeung Shum
Abstract. In this paper, we introduce background cut, a high quality and realtime foreground layer extraction algorithm. From a single video sequence with a moving foreground object and stationary...
Accelerate Video Decoding with Generic GPU (2005)
Guobin Shen, Guang-ping Gao, Shipeng Li, Heung-yeung Shum, Ya-qin Zhang
Abstract—Most modern computers or game consoles are equipped with powerful yet cost-effective graphics processing units (GPUs) to accelerate graphics operations. Though the graphics engines in...
Video object cut and paste (2005)
Yin Li, Jian Sun, Heung-yeung Shum
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage...
Concurrent subspaces analysis (2005)
Dong Xu, Shuicheng Yan, Lei Zhang, Hong-jiang Zhang, Zhengkai Liu, Heung-yeung Shum
A representative subspace is significant for image analysis, while the corresponding techniques often suffer from the curse of dimensionality dilemma. In this paper, we propose a new algorithm,...
This paper focuses on human-computer identification systems against peeping attacks, in which adversaries can observe (and even control) interactions between humans (provers) and computers...
Modeling Hair from Multiple Views (2005)
Yichen Wei, Eyal Ofek, Long Quan, Heung-yeung Shum
Figure 1: From left to right: one of the 40 images captured by a handheld camera under natural conditions; the recovered hair rendered with the recovered diffuse color; a fraction of the longest...
Graduated School of Chinese Academy of (2005)
Jiaping Wang, Xin Tong, John Snyder, Yanyun Chen, Baining Guo, Heung-yeung Shum, ...
∗ This work was done while J. Wang was visiting Microsoft Research Asia. 1
Bi-directional tracking using trajectory segment analysis (2005)
Jian Sun, Weiwei Zhang, Xiaoou Tang, Heung-yeung Shum
Abstract In this paper, we present a novel approach to keyframe-based tracking, called bi-directional tracking. Given two object templates in the beginning and ending keyframes, the bi-directional...
Detecting doctored images using camera response normality and consistency (2005)
Zhouchen Lin, Rongrong Wang, Xiaoou Tang, Heung-yeung Shum
The advance in image/video editing techniques has facilitated people in synthesizing realistic images/videos that may hard to be distinguished from real ones by visual examination. This poses a...
Full-frame video stabilization (2005)
Yasuyuki Matsushita, Eyal Ofek, Xiaoou Tang, Heung-yeung Shum
Video stabilization is an important video enhancement technology which aims at removing annoying shaky motion from videos. We propose a practical and robust approach of video stabilization that...
Video object cut and paste (2005)
Yin Li, Jian Sun, Heung-yeung Shum
Figure 1: Given a video (a) containing an opaque video object on a complicated background, our system can generate clear alpha mattes and foreground colors (b) for the video object, which can be...
Image completion with structure propagation (2005)
Jian Sun, Lu Yuan, Jiaya Jia, Heung-yeung Shum
two intersecting lines (green) specified by the user, (c) intermediate result after propagating structure and texture information along the user-specified lines, and (d) final result after filling in...
Modeling and rendering of quasi-homogeneous materials (2005)
Xin Tong, Jiaping Wang, Stephen Lin, Baining Guo, Heung-yeung Shum
Many translucent materials consist of evenly-distributed heterogeneous elements which produce a complex appearance under different lighting and viewing directions. For these quasi-homogeneous...
Decorating surfaces with bidirectional texture functions (2005)
Kun Zhou, Peng Du, Lifeng Wang, Yasuyuki Matsushita, Jiaoying Shi, Baining Guo, ...
Abstract — We present a system for decorating arbitrary surfaces with bidirectional texture functions (BTF). Our system generates BTFs in two steps. First, we automatically synthesize a BTF over...
Interactive deformation of light fields (2005)
Billy Chen, Eyal Ofek, Heung-yeung Shum, Marc Levoy
Figure 1: Light field deformation enables an animator to interactively deform photo-realistic objects. The left figure shows an image of a light field of a toy Terra Cotta Warrior. The middle image...
Synthesis and Rendering of Bidirectional Texture Functions on Arbitrary Surfaces (2004)
Xinguo Liu, Yaohua Hu, Jingdan Zhang, Xin Tong, Baining Guo, Heung-yeung Shum, ...
Abstract—The bidirectional texture function (BTF) is a 6D function that describes the appearance of a real-world surface as a function of lighting and viewing directions. The BTF can model the...
Jian Sun, Jiaya Jia, Chi-keung Tang, Heung-yeung Shum
∗This research was done when Jiaya Jia was visiting MSR Asia. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided...
Face alignment under variable illumination (2004)
Yuchi Huang, Stephen Lin, Stan Z. Li, Hanqing Lu, Heung-yeung Shum
This paper presents an approach to face alignment under variable illumination, an obstacle largely ignored in previous 2D alignment work. To account for illumination variation, our method employs two...
Example-Based Composite Sketching of Human Portraits (2004)
Hong Chen, Ziqiang Liu, Chuck Rose, Yingqing Xu, Heung-yeung Shum, David Salesin
Corresponding drawings of components in (c); (e) Composite drawing of separate parts as the final drawing. Creating a portrait in the style of a particular artistic tradition or a particular artist...
Radiometric calibration from a single image (2004)
Stephen Lin, Jinwei Gu, Shuntaro Yamazaki, Heung-yeung Shum
Photometric methods in computer vision require calibration of the camera’s radiometric response, and previous works have addressed this problem using multiple registered images captured under...
Face Image Resolution versus Face Recognition Performance Based on Two Global Methods (2004)
Jingdong Wang, Changshui Zhang, Heung-yeung Shum
Face recognition is an interesting topic in computer vision and object recognition. Researchers have proposed numeric recognition methods under the various conditions such as different pose,...
Estimating intrinsic images from image sequences with biased illumination (2004)
Yasuyuki Matsushita, Stephen Lin, Sing Bing Kang, Heung-yeung Shum
Abstract. We present a method for estimating intrinsic images from a fixed-viewpoint image sequence captured under changing illumination directions. Previous work on this problem reduces the...
Mesh editing with poisson-based gradient field manipulation (2004)
Yizhou Yu, Kun Zhou, Dong Xu, Xiaohan Shi, Hujun Bao, Baining Guo, ...
Face alignment under variable illumination (2004)
Yuchi Huang, Stephen Lin, Stan Z. Li, Hanqing Lu, Heung-yeung Shum
This paper presents an approach to face alignment under variable illumination, an obstacle largely ignored in previous 2D alignment work. To account for illumination variation, our method employs two...
Radiometric Calibration from a Single Image (2004)
Stephen Lin Jinwei, Stephen Lin, Jinwei Gu, Shuntaro Yamazaki, Heung-yeung Shum
Photometric methods in computer vision require calibration of the camera's radiometric response, and previous works have addressed this problem using multiple registered images captured under...
Synthesizing dynamic texture with closed-loop linear dynamic system (2004)
Abstract. Dynamic texture can be defined as a temporally continuous and infinitely varying stream of images that exhibit certain temporal statistics. Linear dynamic system (LDS) represented by the...
Bayesian correction of image intensity with spatial consideration (2004)
Jiaya Jia, Jian Sun, Chi-keung Tang, Heung-yeung Shum
Abstract. Under dimly lit condition, it is difficult to take a satisfactory image in long exposure time with a hand-held camera. Despite the use of a tripod, moving objects in the scene still...
Estimating intrinsic images from image sequences with biased illumination (2004)
Yasuyuki Matsushita, Stephen Lin, Sing Bing Kang, Heung-yeung Shum
Abstract. We present a method for estimating intrinsic images from a fixed-viewpoint image sequence captured under changing illumination directions. Previous work on this problem reduces the...
Example-Based Composite Sketching of Human Portraits (2004)
Hong Chen, Ziqiang Liu, Chuck Rose, Yingqing Xu, Heung-yeung Shum, David Salesin
Corresponding drawings of components in (c); (e) Composite drawing of separate parts as the final drawing. Creating a portrait in the style of a particular artistic tradition or a particular artist...
Pop-up light field: An interactive image-based modeling and rendering system (2004)
Heung-yeung Shum, Jian Sun, Yin Li, Chi-keung Tang
In this article, we present an image-based modeling and rendering system, which we call pop-up light field, that models a sparse light field using a set of coherent layers. In our system, the user...
Example-Based Composite Sketching of Human Portraits (2004)
Hong Chen, Ziqiang Liu, Chunk Rose, Yingqing Xu, Heung-yeung Shum, David Salesin
Figure 1: (a)Input image;(b)Decompose image into components;(c)Find best match for each component from training examples; (d)Corresponding drawings of components in (c); (e) Composite drawing of...
Highlight Removal by Illumination-Constrained Inpainting (2003)
Ping Tan, Stephen Lin, Long Quan, Heung-yeung Shum
We present a single-image highlight removal method that incorporates illumination-based constraints into image inpainting. Unlike occluded image regions filled by traditional inpainting, highlight...
Learning to Boost GMM Based Speaker Verification (2003)
Stan Z. Li, Dong Zhang, Chengyuan Ma, Heung-yeung Shum, Eric Chang
The Gaussian mixture models (GMM) has proved to be an effective probabilistic model for speaker verification, and has been widely used in most of state-of-the-art systems. In this paper, we introduce...
A Novel Volume Constrained Smoothing Method for Meshes (2003)
Xinguo Liu Micrpvp, Xinguo Liu, Hujun Bao, Heung-yeung Shum, Qunsheng Peng
this paper weprP ose a novel volume-constrDr smoothing method for trzPQ7Dy7 meshes, whichprpQU2 es exactly the mesh volume dureD the smoothing pr cess, so as toprU ent the mesh frh shrvz2PGD In this...
Combining Shape and Physical Models for On-Line Cursive Handwriting Synthesis (2003)
Jue Wang, Ying-qing Xu, Heung-yeung Shum
This paper proposes a novel learning-based approach to synthesize cursive handwriting of the user's personal handwriting style, by combining shape and physical models together. In the training...
FloatBoost Learning for Classification (2003)
Stan Li Zhenqiu, Stan Z. Li, Heung-yeung Shum, Zhenqiu Zhang Ý, Hongjiang Zhang
AdaBoost [3] minimizes an upper error bound which is an exponential function of the margin on the training set [14]. However, the ultimate goal in applications of pattern classification is always...
Geometry-driven photorealistic facial expression synthesis (2003)
Qingshan Zhang, Zicheng Liu, Baining Guo, Demetri Terzopoulos, Heung-yeung Shum, Senior Member, ...
Abstract—Expression mapping (also called performance driven animation) has been a popular method for generating facial animations. A shortcoming of this method is that it does not generate...
Bi-Scale Radiance Transfer (2003)
Peter-pike Sloan, Xinguo Liu, Heung-yeung Shum, John Snyder
Radiance transfer represents how generic source lighting is shadowed and scattered by an object to produce view-dependent appearance. We generalize by rendering transfer at two scales. A macro-scale...
Image hallucination with primal sketch priors (2003)
Jian Sun, Nan-ning Zheng, Hai Tao, Heung-yeung Shum
In this paper, we propose a Bayesian approach to image hallucination. Given a generic low resolution image, we hallucinate a high resolution image using a set of training images. Our work is inspired...
Highlight Removal by Illumination-Constrained Inpainting (2003)
Ping Tan, Stephen Lin, Long Quan, Heung-yeung Shum
We present a single-image highlight removal method that incorporates illumination-based constraints into image inpainting. Unlike occluded image regions filled by traditional inpainting, highlight...
Stereo matching using belief propagation (2003)
Jian Sun, Heung-yeung Shum, Nan-ning Zheng
Abstract. In this paper, we formulate the stereo matching problem as a Markov network consisting of three coupled Markov random fields (MRF’s). These three MRF’s model a smooth field for...
Realistic rendering and animation of knitwear (2003)
Yanyun Chen, Stephen Lin, Hua Zhong, Ying-qing Xu, Baining Guo, Heung-yeung Shum
We present a framework for knitwear modeling and rendering that accounts for characteristics that are particular to knitted fabrics. We first describe a model for animation that considers knitwear...
Stereo matching using belief propagation (2003)
Jian Sun, Nan-ning Zheng, Heung-yeung Shum
In this paper, we formulate the stereo matching problem as a Markov network and solve it using Bayesian belief propagation. The stereo Markov network consists of three coupled Markov random fields...
Multiple-cue illumination estimation in textured scenes (2003)
Yuanzhen Li, Stephen Lin, Hanqing Lu, Heung-yeung Shum
In this paper, we present a method that integrates cues from shading, shadow and specular reflections for estimating directional illumination in a textured scene. Texture poses a problem for lighting...
Subdivision Surface Simplification (2002)
Jeong, Won-Ki, Kähler, Kolja, Haber, Jörg, Seidel, Hans-Peter, Coquillart, Sabine, Shum, Heung-Yeung, ...
Interactive Rendering of Translucent Objects (2002)
Lensch, Hendrik P. A., Goesele, Michael, Bekaert, Philippe, Kautz, Jan, Magnor, Marcus, Lang, Jochen, ...
This paper presents a rendering method for translucent objects, in which view point and illumination can be modified at interactive rates. In a preprocessing step the impulse response to incoming...
A Simple Approach to Interactive Free-Form Shape Deformations (2002)
Yoshizawa, Shin, Belyaev, Alexander, Seidel, Hans-Peter, Coquillart, Sabine, Shum, Heung-Yeung, Hu, Shi-Min
In this paper, we propose a set of free-form shape deformation techniques. The basic technique can be described as follows. Given a surface represented by a mesh and a control point, for every mesh...
Theobalt, Christian, Magnor, Marcus, Schüler, Pascal, Seidel, Hans-Peter, Coquillart, Sabine, Shum, Heung-Yeung, ...
A Divide and Conquer Algorithm for Triangle Mesh Connectivity Encoding (2002)
Ivrissimtzis, Ioannis, Rössl, Christian, Seidel, Hans-Peter, Coquillart, Sabine, Shum, Heung-Yeung, Hu, Shi-Min
We propose a divide and conquer algorithm for the single resolution encoding of triangle mesh connectivity. Starting from a boundary edge we grow a zig-zag strip which divides the mesh into two...
Theobalt, Christian, Magnor, Marcus, Schüler, Pascal, Seidel, Hans-Peter, Coquillart, Sabine, Shum, Heung-Yeung, ...
Interactive Rendering of Translucent Objects (2002)
Lensch, Hendrik, Gösele, Michael, Bekaert, Philippe, Kautz, Jan, Magnor, Marcus, Lang, Jochen, ...
Modeling and rendering of realistic feathers (2002)
Yanyun Chen, Yingqing Xu, Baining Guo, Heung-yeung Shum
We present techniques for realistic modeling and rendering of feathers and birds. Our approach is motivated by the observation that a feather is a branching structure that can be described by an...
Hierarchical shape modeling for automatic face localization (2002)
Ce Liu, Heung-yeung Shum, Changshui Zhang
Abstract. Many approaches have been proposed to locate faces in an image. There are, however, two problems in previous facial shape models using feature points. First, the dimension of the solution...
Example-based automatic portraiture (2002)
Hong Chen, Lin Liang, Ying-qing Xu, Heung-yeung Shum, Nan-ning Zheng
In this paper, we present an example-based approach for automatically generating a life-like portrait from a frontal face image. Based on an inhomogeneous Markov Random Field Model, an inhomogeneous...
Motion texture: A two-level statistical model for character motion synthesis (2002)
Yan Li, Tianshu Wang, Heung-yeung Shum
Figure 1: This 320-frame sequence of dance motion is choreographed from (1) the starting frame, (2) the ending frame and (3) the learnt motion texture from motion captured dance data. Four motion...
Learning-based cursive handwriting synthesis (2002)
Jue Wang, Chenyu Wu, Ying-qing Xu, Heung-yeung Shum, Liang Ji
In this paper, an integrated approach for modeling, learning and synthesizing personal cursive handwriting is proposed. Cursive handwriting is modeled by a tri-unit handwriting model, which focuses...
Pattern-based texture metamorphosis (2002)
Ziqiang Liu, Ce Liu, Heung-yeung Shum, Yizhou Yu
Figure 1. A texture metamorphosis sequence is generated from the leftmost to the rightmost. In this paper, we study texture metamorphosis, or how to generate texture samples that smoothly transform...
Learning-based cursive handwriting synthesis (2002)
Jue Wang, Chenyu Wu, Ying-qing Xu, Heung-yeung Shum, Liang Ji
In this paper, an integrated approach for modeling, learning and synthesizing personal cursive handwriting is proposed. Cursive handwriting is modeled by a tri-unit handwriting model, which focuses...
Diffuse-specular separation and depth recovery from image sequences (2002)
Stephen Lin, Yuanzhen Li, Sing Bing Kang, Xin Tong, Heung-yeung Shum
Abstract. Specular reflections present difficulties for many areas of computer vision such as stereo and segmentation. To separate specular and diffuse reflection components, previous approaches...
Learning a highly structured motion model for 3D human tracking (2002)
Tao Zhao, Tianshu Wang, Heung-yeung Shum
This paper presents our work on learning high level structure from human motion sequences, and its applications in human figure tracking. We use a structured representation (“primitives ” and...
A novel probability model for background maintenance and subtraction (2002)
Dongsheng Wang, Tao Feng, Heung-yeung Shum, Songde Ma
“Background maintenance and subtraction ” is an important problem for many computer vision applications. This paper proposes a novel model for background. This model includes two components and...
Rendering by Manifold Hopping (2002)
Heung-yeung Shum, Lifeng Wang, Jin-xiang Chai, Xin Tong
In this paper, we present a novel image-based rendering technique, which we call manifold hopping. Our technique provides users with perceptually continuous navigation by using only a small number of...
A novel volume constrained smoothing method for meshes (2002)
Xinguo Liu, Hujun Bao, Heung-Yeung Shum, Qunsheng Peng
In general, mesh smoothing is performed by minimizing the discrete energy function for the surface. One of the major problems in mesh smoothing is to prevent the mesh from shrinking. In this paper,...
Interacting with 3D Graphic Objects in (2002)
Yong Wang, Jiang Li, Kun Zhou, Heung-yeung Shum
This paper presents a novel hybrid rendering system, which allows users to interact with 3D graphic objects in an image-based environment. To represent a complex environment, we use a collection of...
Xinguo Liu, Hujun Bao, Heung-yeung Shum, Qunsheng Peng
In general, mesh smoothing is performed by minimizing the discrete energy function for the surface. One of the major problems in mesh smoothing is to prevent the mesh from shrinking. In this paper,...
Synthesis of bidirectional texture functions on arbitrary surfaces (2002)
Xin Tong, Jingdan Zhangþ, Ligang Liu, Xi Wangþ, Baining Guo, Heung-yeung Shum
The bidirectional texture function (BTF) is a 6D function that can describe textures arising from both spatially-variant surface reflectance and surface mesostructures. In this paper, we present an...
Example-based caricature generation with exaggeration (2002)
Lin Liang, Hong Chen, Ying-qing Xu, Heung-yeung Shum
In this paper, we present a system that automatically generates caricatures from input face images. From example caricatures drawn by an artist, our caricature system learns how an artist draws...
Motion texture: A two-level statistical model for character motion synthesis (2002)
Yan Li, Tianshu Wang, Heung-yeung Shum
Figure 1: This 320-frame sequence of dance motion is choreographed from (1) the starting frame, (2) the ending frame and (3) the learnt motion texture from motion captured dance data. Four motion...
Pattern-based texture metamorphosis (2002)
Ziqiang Liu, Ce Liu, Heung-yeung Shum, Yizhou Yu
Figure 1. A texture metamorphosis sequence is generated from the leftmost to the rightmost. In this paper, we study texture metamorphosis, or how to generate texture samples that smoothly transform...
Concentric mosaics, planar motions and 1d cameras (2001)
Long Quan, Le Lu, Heung-yeung Shum, Maxime Lhuillier
General SFM methods give poor results for images captured by constrained motions such as planar motion of concentric mosaics (CM). In this paper, we propose new SFM algorithms for both images...
Example-Based Facial Sketch Generation with Non-Parametric Sampling (2001)
Hong Chen, Ying-qing Xu, Heung-yeung Shum, Song-chun Zhu, Nan-ning Zheng
In this paper, we present an example-based facial sketch system. Our system automatically generates a sketch from an input image, by learning from example sketches drawn with a particular style by an...
Asymptotically admissible texture synthesis (2001)
Yingqing Xu, Song-chun Zhu, Baining Guo, Heung-yeung Shum
Recently there is a resurgent interest in example based texture analysis and synthesis in both computer vision and computer graphics. While study in computer vision is concerned with learning...
Jin Li, Heung-yeung Shum, Ya-qin Zhang
In image based rendering (IBR), a 3D scene is recorded through a set of photos which is then rendered to form novel views. Compression is essential to reduce the huge data amount of IBR. In this...
Efficient dense depth estimation from dense multiperspective panoramas (2001)
Yin Li, Chi-keung Tang, Heung-yeung Shum
In this paper we study how to compute a dense depth map with panoramic field of view (e.g., £¥¤¥ ¦ degrees) from multiperspective panoramas. A dense sequence of multiperspective panoramas is...
Speech-Driven Cartoon Animation with Emotions (2001)
Yan Li, Feng Yu, Ying-qing Xu, Eric Chang, Heung-yeung Shum
In this paper, we present a cartoon face animation system for multimedia HCI applications. We animate face cartoons not only from input speech, but also based on emotions derived from speech signal....
Error analysis of pure rotation-based self-calibration (2001)
Lei Wang, Sing Bing Kang, Heung-yeung Shum, Guangyou Xu
Self-calibration using pure rotation is a well-known technique, and has been shown to be a reliable means for recovering intrinsic camera parameters. However, in practice, it is virtually impossible...
Efficient dense depth estimation from dense multiperspective panoramas (2001)
Yin Li, Chi-keung Tang, Heung-yeung Shum
In this paper we study how to compute a dense depth map with panoramic field of view (e.g., 360 degrees) from multiperspective panoramas. A dense sequence of multiperspective panoramas is used for...
Example-Based Facial Sketch Generation with Non-Parametric Sampling (2001)
Hong Chen, Ying-qing Xu, Heung-yeung Shum, Song-chun Zhu, Nan-ning Zheng
In this paper, we present an example-based facial sketch system. Our system automatically generates a sketch from an input image, by learning from example sketches drawn with a particular style by an...
Real-time texture synthesis by patch-based sampling (2001)
Lin Liang, Ce Liu, Yingqing Xu, Baining Guo, Heung-yeung Shum
We present a patch-based sampling algorithm for synthesizing textures from an input sample texture. The patch-based sampling algorithm is fast. Using patches of the sample texture as building blocks...
Emotion detection from speech to enrich multimedia content (2001)
Feng Yu, Eric Chang, Ying-qing Xu, Heung-yeung Shum
The paper describes an experimental study on the detection of emotion from speech. As computer based characters such as avatars and virtual chat faces become more common, the use of emotion to drive...
Concentric mosaics, planar motions and 1d cameras (2001)
Long Quan, Le Lu, Heung-yeung Shum, Maxime Lhuillier
General SFM methods give poor results for images captured by constrained motions such as planar motion of concentric mosaics (CM). In this paper, we propose new SFM algorithms for both images...
Gang Xu, Jun-ichi Terai, Heung-yeung Shum
In this paper we show that given two homography matrices for two planes in space, there is a linear algorithm for the rotation and translation between the two cameras, the focal lengths of the two...
Gang Xu, Jun-ichi Terai, Heung-yeung Shum
yt #Y$R }s}## b# RYBb #Ys# #$%#t #bB YB8B##s}Yk 8s##${#R PB# #bB }jst#R $t R}s{## #Y### $R s j$t#s# sj#BT #$#Y8 PB# #Y# #B#s#$Bt stI ##stRjs#$Bt A##b##t #Y# #bB {s8##sR# #Y# PB{sj j#t##YR BP #Y# #bB...
Rendering with non-uniform approximate concentric mosaics (2000)
Jinxiang Chai, Sing Bing Kang, Heung-yeung Shum
Abstract. In this paper, we explore the more practical aspects of building and rendering concentric mosaics. First, we use images captured with only approximately circular camera trajectories. The...
Ying-qing Xu, Chiyi Cheng, Jiaoying Shi, Heung-yeung Shum
We propose a novel model for curtain real-time animation, which not only significantly reduces the time of rendering, but is also able to maintain visually appealing results. Conventional physically...
Virtual reality using the concentric mosaic: construction, rendering and data compression (2000)
Heung-yeung Shum, King-to Ng, Shing-chow Chan
This paper proposes a new image based rendering technique called concentric mosaic for virtual reality applications. It is constructed by capturing vertical slit images when a camera is moving around...
Interactive shape from shading (1999)
Gang Zeng, Yasuyuki Matsushita, Long Quan, Heung-yeung Shum
Shape from shading (SfS) has always been difficult for real applications due to its intrinsic ill-posedness. In this paper, we propose an interactive SfS method which efficiently uses human knowledge...
A survey of image-based rendering techniques (1999)
Heung-yeung Shum, Sing Bing Kang
In this paper, we survey the techniques for image-based rendering. Unlike traditional 3D computer graphics in which 3D geometry of the scene is known, image-based rendering techniques render novel...
Rendering with Concentric Mosaics (1999)
This paper presents a novel 3D plenoptic function, which we call concentric mosaics. We constrain camera motion to planar concentric circles, and create concentric mosaics using a manifold mosaic for...
Heung-yeung Shum, Adam Kalai, Steven M. Seitz
The notion of a virtual sensor for optimal 3D reconstruction is introduced. Instead of planar perspective images that collect many rays at a fixed viewpoint, omnivergent cameras collect a small...
Interactive Shape from Shading (1999)
Gang Zeng Yasuyuki, Gang Zeng, Yasuyuki Matsushita, Long Quan, Heung-yeung Shum
Shape from shading (SfS) has always been difficult for real applications due to its intrinsic ill-posedness. In this paper, we propose an interactive SfS method which efficiently uses human knowledge...
A survey of image-based rendering techniques (1999)
Heung-yeung Shum, Sing Bing Kang
In this paper, we survey the techniques for image-based rendering. Unlike traditional 3D computer graphics in which 3D geometry of the scene is known, image-based rendering techniques render novel...
Adaptive Control of Space Robot System with an Attitude Controlled Base, (1998)
Xu, Yangsheng, Shum, Heung-Yeung, Lee, Ju-Jang, Kanade, Takeo
In this report, we discuss adaptive control of a space robot system with an attitude controlled base on which the robot is attached. We at first derive the system kinematic and dynamic equations...
Dynamic Control of a Space Robot System With No Thrust Jets Controlled Base. (1998)
Xu, Yangsheng, Shum, Heung-Yeung
In this paper we discuss dynamic control of a free-flying space robot system where the base attitude is not controlled by thrust jets. Without external forces and moments, the system is governed by...
Shum, Heung-Yeung, Hebert, Martial, Ikeuchi, Katsushi
We present a novel approach to 3D shape synthesis of closed surfaces. A curved or polyhedral 3D object of genus zero is represented by a curvature distribution on a spherical mesh that has nearly...
On 3D Shape Similarity, (1998)
Shum, Heung-Yeung, Hebert, Martial, Ikeuchi, Katsushi
We study the 3D shape similarity between closed surfaces. We represent a curved or polyhedral 3D object of genus zero using a mesh representation that has nearly uniform distribution with known...
Interactive 3d modeling from multiple images using scene regularities (1998)
Heung-yeung Shum, Richard Szeliski, Simon Baker, Mei Han, P. An
Due to the complexity of real scenes and the fragility of fully automated vision techniques, results from many automated modeling systems are disappointing. Automated techniques often require manual...
Interactive construction of 3d models from panoramic mosaics (1998)
Heung-yeung Shum, Mei Han, Richard Szeliski
This paper presents an interactive modeling system that constructs 3D models from a collection of panoramic image mosaics. A panoramic mosaic consists of a set of images taken around the same...
Interactive 3d modeling from multiple images using scene regularities (1998)
Heung-yeung Shum, Richard Szeliski, Simon Baker, Mei Han, P. An
Abstract. We present some recent progress in designing and implementing two interactive image-based 3D modeling systems. The first system constructs 3D models from a collection of panoramic image...
Interactive Construction of 3D Models from Panoramic Mosaics (1998)
This paper presents an interactive modeling system that constructs 3D models from a collection of panoramic image mosaics. A panoramic mosaic consists of a set of images taken around the same...
Interactive construction of 3d models from panoramic mosaics (1998)
Heung-yeung Shum, Mei Han, Richard Szeliski
This paper presents an interactive modeling system that constructs 3D models from a collection of panoramic image mosaics. A panoramic mosaic consists of a set of images taken around the same...
Panoramic Image Mosaics (1997)
Heung-yeung Shum, Heung-yeung Shum, Richard Szeliski, Richard Szeliski
This paper presents some techniques for constructing panoramic image mosaics from sequences of images. Our mosaic representation associates a transformation matrix with each input image, rather than...
Creating Full View Panoramic Image Mosaics and Environment Maps (1997)
Richard Szeliski, Heung-Yeung Shum
This paper presents a novel approach to creating full view panoramic mosaics from image sequences. Unlike current panoramic stitching methods, which usually require pure horizontal camera panning,...
Panoramic Image Mosaics (1997)
Heung-yeung Shum, Heung-yeung Shum, Richard Szeliski, Richard Szeliski
This paper presents some techniques for constructing panoramic image mosaics from sequences of images. Our mosaic representation associates a transformation matrix with each input image, rather than...
Heung-yeung Shum, Martial Hebert, Katsushi Ikeuchi
conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies or endorsements, either expressed or implied, of the Department of...
Modeling from reality: representation and integration (1996)
Heung-yeung Shum, Heung-yeung Shum
not be interpreted as necessarily
Heung-yeung Shum, Martial Hebert, Katsushi Ikeuchi
the official policies or endorsements, either expressed or implied, of the Department of the Army, or the U.S. government.
Heung-yeung Shum, Martial Hebert, Katsushi Ikeuchi
the official policies or endorsements, either expressed or implied, of the Department of the Army, or the U.S. government.
Motion estimation with quadtree splines (1996)
Richard Szeliski, Richard Szeliski, Heung-yeung Shum, Heung-yeung Shum
The Cambridge laboratory became operational in 1988 and is located at One Kendall Square, near MIT. CRL engages in computing research to extend the state of the computing art in areas likely to be...
Motion Estimation with Quadtree Splines (1995)
Richard Szeliski, Richard Szeliski, Heung-yeung Shum, Heung-yeung Shum
This paper presents a motion estimation algorithm based on a new multiresolution representation, the quadtree spline. This representation describes the motion field as a collection of smoothly...
A Parallel Feature Tracker for Extended Image sequences (1995)
Richard Szeliski, Richard Szeliski, Sing Bing Kang, Sing Bing Kang, Heung-yeung Shum, Heung-yeung Shum
This paper presents a feature tracker for long image sequences based on simultaneously estimating the motions and deformations of a collection of adjacent image patches. By sharing common corner...
Heung-yeung Shum, Martial Hebert, Katsushi Ikeuchi
We study the 3D shape similarity between closed surfaces. We represent a curved or polyhedral 3D object of genus zero using a mesh representation that has nearly uniform distribution with known...
A parallel feature tracker for extended image sequences (1995)
Heung-yeung Shum, Richard Szeliski, Richard Szeliski, Sing Bing Kang, Sing Bing Kang
The Cambridge laboratory became operational in 1988 and is located at One Kendall Square, near MIT. CRL engages in computing research to extend the state of the computing art in areas likely to be...
A parallel feature tracker for extended image sequences (1995)
Sing Bing Kang, Richard Szeliski, Heung-yeung Shum
This paper presents a feature tracker for long image sequences based on simultaneously estimating the motions and deformations of a collection of adjacent image patches. By sharing common corner...
Parameterization and Adaptive Control of Space Robot Systems (1995)
Xu, Yangsheng, Shum, Heung-Yeung, Kanade, Takeo, Lee, Ju-Jang
In space application, robot system are subject to unknown or unmodeled dynamics, for example, in the tasks of transporting an unknown payload or catching an unmodeled moving object. We discuss the...
Adaptive motion and force control of robot manipulators with uncertainties / (1990)
Thesis (M. Phil.)--University of Hong Kong, 1991.
Adaptive motion and force control of robot manipulators with uncertainties / (1990)
Thesis (M. Phil.)--University of Hong Kong, 1991.
Adaptive motion and force control of robot manipulators with uncertainties / (1990)
Thesis (M. Phil.)--University of Hong Kong, 1991.
Thesis (M.Phil.)--University of Hong Kong, 1991.