The Nyström method is a well-known sampling-based technique for approximating the eigensystem of large kernel matrices. However, the chosen samples in the Nyström method are all assumed to be of...
Improved Nyström Low-Rank Approximation and Error Analysis (2009)
Kai Zhang, Ivor W. Tsang, James T. Kwok
Low-rank matrix approximation is an effective tool in alleviating the memory and computational burdens of kernel methods and sampling, as the mainstream of such algorithms, has drawn considerable...
The Nyström method is a well-known sampling-based technique for approximating the eigen-system of large kernel matrices. However, the chosen samples in the Nyström method are all assumed to be of...
Transferring Localization Models Across Space (2009)
Sinno Jialin Pan, Dou Shen, Qiang Yang, James T. Kwok
Machine learning approaches to indoor WiFi localization involve an offline phase and an online phase. In the offline phase, data are collected from an environment to build a localization model, which...
Ensembles of Partially Trained SVMs with Multiplicative Updates (2008)
The training of support vector machines (SVM) involves a quadratic programming problem, which is often optimized by a complicated numerical solver. In this paper, we propose a much simpler approach...
A Class of Single-Class Minimax Probability Machines for Novelty Detection (2008)
James T. Kwok, Jacek M. Zurada
Abstract—Single-class minimax probability machines (MPMs) offer robust novelty detection with distribution-free worst case bounds on the probability that a pattern will fall inside the normal...
FAST SPEAKER ADAPTION VIA MAXIMUM PENALIZED LIKELIHOOD KERNEL REGRESSION (2008)
Ivor W. Tsang, James T. Kwok, Brian Mak, Kai Zhang, Jeffrey J. Pan
Maximum likelihood linear regression (MLLR) has been a popular speaker adaptation method for many years. In this paper, we investigate a generalization of MLLR using nonlinear regression....
DOI 10.1007/s10994-006-6130-8 Model-based transductive learning of the kernel matrix (2008)
Zhihua Zhang, James T. Kwok, Dit-yan Yeung, Z. Zhang, J. T. Kwok
Abstract This paper addresses the problem of transductive learning of the kernel matrix from a probabilistic perspective. We define the kernel matrix as a Wishart process prior and construct a...
Ensembles of Partially Trained SVMs with Multiplicative Updates (2008)
The training of support vector machines (SVM) involves a quadratic programming problem, which is often optimized by a complicated numerical solver. In this paper, we propose a much simpler approach...
Simpler Core Vector Machines with Enclosing Balls (2008)
Ivor W. Tsang, Andras Kocsor, James T. Kwok
The core vector machine (CVM) is a recent approach for scaling up kernel methods based on the notion of minimum enclosing ball (MEB). Though conceptually simple, an efficient implementation still...
Data-Dependent Kernels for High-Dimensional Data Classification (2008)
Jingdong Wang, James T. Kwok, Helen C. Shen, Long Quan
Abstract — For high-dimensional data classification problems such as face recognition, one of the most efficient classifiers is the Nearest Neighbor (NN) classifier. What mostly affects the NN...
LETTER Communicated by Klaus-Robert Müller SVDD-Based Pattern Denoising (2008)
Jooyoung Park, Daesung Kang, Jongho Kim, James T. Kwok, Ivor W. Tsang
The support vector data description (SVDD) is one of the best-known one-class support vector learning methods, in which one tries the strategy of using balls defined on the feature space in order to...
Locally Adaptive Classification Piloted by Uncertainty (2008)
Juan Dai, Shuicheng Yan, Xiaoou Tang, James T. Kwok
Locally adaptive classifiers are usually superior to the use of a single global classifier. However, there are two major problems in designing locally adaptive classifiers. First, how to place the...
Locally Adaptive Classification Piloted by Uncertainty (2008)
Juan Dai, Shuicheng Yan, Xiaoou Tang, James T. Kwok
Locally adaptive classifiers are usually superior to the use of a single global classifier. However, there are two major problems in designing locally adaptive classifiers. First, how to place the...
Multimodal Registration using the Discrete Wavelet Frame Transform (2008)
Shutao Li, Jinglin Peng, James T. Kwok, Jing Zhang
Image registration is a critical step in medical image analysis. In this paper, a novel image registration method based on the discrete wavelet frame transform (DWFT) and the sum of absolute distance...
Maximum Margin Clustering Made Practical (2008)
Kai Zhang, Ivor W. Tsang, James T. Kwok
Maximum margin clustering (MMC) is a recent large margin unsupervised learning approach that has often outperformed conventional clustering methods. Computationally, it involves non-convex...
Efficient Hyperkernel Learning Using (2008)
1 Introduction In recent years, kernels have been successfully used in various aspects of machine learning, such as classification, regression, clustering, ranking and principal component analysis...
FAST SPEAKER ADAPTION VIA MAXIMUM PENALIZED LIKELIHOOD KERNEL REGRESSION (2008)
Ivor W. Tsang, James T. Kwok, Brian Mak, Kai Zhang, Jeffrey J. Pan
Maximum likelihood linear regression (MLLR) has been a popular speaker adaptation method for many years. In this paper, we investigate a generalization of MLLR using nonlinear regression....
Zhihua Zhang, Kap Luk Chan, James T. Kwok, Dit-yan Yeung
Based on the probabilistic reformulation of principal component analysis (PCA), we consider the problem of determining the number of principal components as a model selection problem. We present a...
Maximum Margin Clustering Made Practical (2008)
Kai Zhang, Ivor W. Tsang, James T. Kwok
Maximum margin clustering (MMC) is a recent large margin unsupervised learning approach that has often outperformed conventional clustering methods. Computationally, it involves non-convex...
Ivor W. Tsang, James T. Kwok, Clear Water Bay
Recently, Loosli and Canu (2007) reported that the core vector machine (CVM) (Tsang et al., 2005) becomes unstable when C is large. We investigated this problem and found that there are at least two...
Simpler Core Vector Machines with Enclosing Balls (2008)
Ivor W. Tsang, Andras Kocsor, James T. Kwok
The core vector machine (CVM) is a recent approach for scaling up kernel methods based on the notion of minimum enclosing ball (MEB). Though conceptually simple, an efficient implementation still...
Adaptive Localization in a Dynamic WiFi Environment Through Multi-view Learning ∗ (2008)
Sinno Jialin Pan, James T. Kwok, Qiang Yang, Jeffrey Junfeng Pan
Accurately locating users in a wireless environment is an important task for many pervasive computing and AI applications, such as activity recognition. In a WiFi environment, a mobile device can be...
Zhihua Zhang, James T. Kwok, Dit-Yan Yeung, Gang Wang
In this paper, we propose a family of surrogate maximization (SM) algorithms for multi-class logistic regression models (also called conditional exponential models). An SM algorithm aims at turning...
A Novel Distance-based Classifier Using Convolution Kernels and Euclidean (2007)
Zhihua Zhang, James T. Kwok, Dit-yan Yeung, Wanqiu Wang
Distance-based classification methods such as the nearest-neighbor and k-nearest-neighbor classifiers have to rely on a metric or distance measure between points in the input space. For many...
A Novel Family of Subspace Methods---Protoface and Its Kernel Version (2007)
Zhihua Zhang, James T. Kwok, Dit-yan Yeung, Wanqiu Wang
In this paper, we present a novel feature extraction called the protoface. While the Eigenface is based on principal component analysis and the Fisherface on Fisher's linear discriminant...
Linear Dependency Between ffl and The Input Noise in ffl-Support Vector Regression (2007)
Abstract. In using the ffl-support vector regression (ffl-SVR) algorithm, one has to decide on a suitable value of the insensitivity parameter ffl. Smola et al. [6] determined its "optimal...
Applying the Bayesian Evidence Framework to -Support Vector Regression (2007)
Martin H. Law, James T. Kwok, Clear Water Bay
Abstract. Following previous successes on applying the Bayesian evidence framework to support vector classiers and the -support vector regression algorithm, in this paper we extend the evidence...
Hailong Zhu, James T. Kwok, Liangsheng Qu
Soft thresholding has been a standard wavelet de-noising procedure in many signal and image processing applications. Theoretically, it is also almost optimal in the sense of nearly achieving the...
Multifocus Image Fusion using Artificial Neural Networks (2007)
Shutao Li James, James T. Kwok, Yaonan Wang, Clear Water Bay
Optical lenses, particularly those with long focal lengths, suffer from the problem of limited depth of field. Consequently, it is often difficult to obtain good focus for all objects in the picture....
Probabilistic Kernel Matrix Learning with a Mixture Model of Kernels (2007)
Zhihua Zhang, Dit-yan Yeung, James T. Kwok
This paper addresses the kernel matrix learning problem in kernel methods. We model the kernel matrix as a random positive definite matrix following the Wishart distribution, with the parameter...
Sliced coordinate analysis for effective dimension reduction and nonlinear extensions (2007)
Zhang, Zhihua, Yeung, Dit-Yan, Kwok, James T., Chang, Edward Y.
Sliced inverse regression (SIR) is an important method for reducing the dimensionality of input variables. Its goal is to estimate the effective dimension reduction directions. In classification...
Surrogate maximization/minimization algorithms and extensions (2007)
Zhang, Zhihua, Kwok, James T., Yeung, Dit-Yan
Surrogate maximization (or minimization) (SM) algorithms are a family of algorithms that can be regarded as a generalization of expectation-maximization (EM) algorithms. An SM algorithm aims at...
Surrogate maximization/minimization algorithms and extensions (2007)
Zhang, Zhihua, Kwok, James T., Yeung, Dit-Yan
Surrogate maximization (or minimization) (SM) algorithms are a family of algorithms that can be regarded as a generalization of expectation-maximization (EM) algorithms. An SM algorithm aims at...
Sliced Coordinate Analysis for Effective Dimension Reduction and Nonlinear Extensions (2007)
Zhihua Zhang, Dit-yan Yeung, James T. Kwok, Edward Y. Chang
Sliced inverse regression (SIR) is an important method for reducing the dimensionality of input variables. Its goal is to estimate the effective dimension reduction directions. In classification...
Marginalized multi-instance kernels (2007)
James T. Kwok, Pak-ming Cheung
Support vector machines (SVM) have been highly successful in many machine learning problems. Recently, it is also used for multi-instance (MI) learning by employing a kernel that is defined directly...
Recently, [4] reported that the CVM [6] becomes unstable when C is large. We investigated this problem and found that there are at least two factors for this: 1. The binary for the Linux version is...
Model-based transductive learning of the kernel matrix (2006)
Zhihua Zhang, James T. Kwok, Dit-yan Yeung
This paper addresses the problem of transductive learning of the kernel matrix from a probabilistic per-spective. We define the kernel matrix as a Wishart process prior and construct a hierarchical...
Block-quantized kernel matrix for fast spectral embedding (2006)
Eigendecomposition of kernel matrix is an indispensable procedure in many learning and vision tasks. However, the cubic complexity O(N 3) is impractical for large problem, where N is the data size....
A regularization framework for multiple-instance learning (2006)
Pak-ming Cheung, James T. Kwok
This paper focuses on kernel methods for multi-instance learning. Existing methods require the prediction of the bag to be identical to the maximum of those of its individual instances. However, this...
Simplifying mixture models through function approximation (2006)
Finite mixture model is a powerful tool in many statistical learning problems. In this paper, we propose a general, structure-preserving approach to reduce its model complexity, which can bring...
Large-scale sparsified manifold regularization (2006)
Semi-supervised learning is more powerful than supervised learning by using both labeled and unlabeled data. In particular, the manifold regularization framework, together with kernel methods, leads...
A regularization framework for multiple-instance learning (2006)
Pak-ming Cheung, James T. Kwok
This paper focuses on kernel methods for multi-instance learning. Existing methods require the prediction of the bag to be identical to the maximum of those of its individual instances. However, this...
A novel incremental principal component analysis and its application for face recognition (2006)
Haitao Zhao, Pong Chi Yuen, James T. Kwok
Abstract—Principal component analysis (PCA) has been proven to be an efficient method in pattern recognition and image analysis. Recently, PCA has been extensively employed for facerecognition...
Jeffrey Junfeng Pan, James T. Kwok, Qiang Yang, Senior Member, Yiqiang Chen
Abstract—In this paper, we present an algorithm for multidimensional vector regression on data that are highly uncertain and nonlinear, and then apply it to the problem of indoor location...
Block-quantized kernel matrix for fast spectral embedding (2006)
Eigendecomposition of kernel matrix is an indispensable procedure in many learning and vision tasks. However, the cubic complexity O(N 3) is impractical for large problem, where N is the data size....
Simplifying mixture models through function approximation (2006)
The finite mixture model is widely used in various statistical learning problems. However, the model obtained may contain a large number of components, making it inefficient in practical...
Core vector regression for very large regression problems (2005)
Ivor W. Tsang, James T. Kwok, Kimo T. Lai
In this paper, we extend the recently proposed Core Vector Machine algorithm to the regression setting by generalizing the underlying minimum enclosing ball problem. The resultant Core Vector...
Very Large SVM Training using Core Vector Machines (2005)
Ivor Tsang James, James T. Kwok, Clear Water Bay
Standard SVM training has O(m ) time and O(m ) space complexities, where m is the training set size. In this paper, we scale up kernel methods by exploiting the "approximateness" in...
Abstract—Recently, we proposed an improvement to the conventional eigenvoice (EV) speaker adaptation using kernel methods. In our novel kernel eigenvoice (KEV) speaker adaptation, speaker...
J.T.: Applying neighborhood consistency for fast clustering and kernel density estimation (2005)
Kai Zhang, Ming Tang, James T. Kwok
Nearest neighborhood consistency is an important concept in statistical pattern recognition, which underlies the well-known k-nearest neighbor method. In this paper, we combine this idea with kernel...
Core vector machines: Fast SVM training on very large data sets (2005)
Ivor W. Tsang, James T. Kwok, Pak-ming Cheung, Nello Cristianini
Standard SVM training has O(m 3) time and O(m 2) space complexities, where m is the training set size. It is thus computationally infeasible on very large data sets. By observing that practical SVM...
Kernel relevant component analysis for distance metric learning (2005)
Ivor W. Tsang, Pak-ming Cheung, James T. Kwok
Abstract — Defining a good distance measure between patterns is of crucial importance in many classification and clustering algorithms. Recently, relevant component analysis (RCA) is proposed which...
Core vector regression for very large regression problems (2005)
Ivor W. Tsang, James T. Kwok, Kimo T. Lai
In this paper, we extend the recently proposed Core Vector Machine algorithm to the regression setting by generalizing the underlying minimum enclosing ball problem. The resultant Core Vector...
Core vector regression for very large regression problems (2005)
Ivor W. Tsang, James T. Kwok, Kimo T. Lai
In this paper, we extend the recently proposed Core Vector Machine algorithm to the regression setting by generalizing the underlying minimum enclosing ball problem. The resultant Core Vector...
J.T.: Applying neighborhood consistency for fast clustering and kernel density estimation (2005)
Kai Zhang, Ming Tang, James T. Kwok
Nearest neighborhood consistency is an important concept in statistical pattern recognition, which underlies the well-known k-nearest neighbor method. In this paper, we combine this idea with kernel...
Zhihua Zhang, Dit-yan Yeung, James T. Kwok
In kernel methods, an interesting recent development seeks to learn a good kernel from empirical data automatically. In this paper, by regarding the transductive learning of the kernel matrix as a...
Surrogate maximization/minimization algorithms for adaboost and the logistic regression model (2004)
Zhihua Zhang, James T. Kwok, Dit-yan Yeung
Surrogate maximization (or minimization) (SM) algorithms are a family of algorithms that can be regarded as a generalization of expectation-maximization (EM) algorithms. There are three major...
Surrogate maximization/minimization algorithms for adaboost and the logistic regression model (2004)
Zhihua Zhang, James T. Kwok, Dit-yan Yeung
Surrogate maximization (or minimization) (SM) algorithms are a family of algorithms that can be regarded as a generalization of expectation-maximization (EM) algorithms. An SM algo-rithm aims at...
Eigenvoice speaker adaptation via composite kernel PCA (2004)
James T. Kwok, Brian Mak, Simon Ho
Eigenvoice speaker adaptation has been shown to be effective when only a small amount of adaptation data is available. At the heart of the method is principal component analysis (PCA) employed to...
Speedup of kernel eigenvoice speaker adaptation by embedded kernel PCA (2004)
Brian Mak, Simon Ho, James T. Kwok
Recently, we proposed an improvement to the eigenvoice (EV) speaker adaptation called kernel eigenvoice (KEV) speaker adaptation. In KEV adaptation, eigenvoices are computed using kernel PCA, and a...
Surrogate maximization/minimization algorithms for adaboost and the logistic regression model (2004)
Zhihua Zhang, James T. Kwok, Dit-yan Yeung, Z. Zhang
Abstract Surrogate maximization (or minimization) (SM) algorithms are a family of algorithms that can be regarded as a generalization of expectation-maximization (EM) algorithms. An SM algorithm aims...
Parametric distance metric learning with label information (2003)
Zhihua Zhang, James T. Kwok, Dit-yan Yeung
Distance-based methods in pattern recognition and machine learning have to rely on a similarity or dissimilarity measure between patterns in the input space. For many applications, Euclidean distance...
Parametric Distance Metric Learning with Label Information (2003)
Zhihua Zhang James, James T. Kwok, Dit-yan Yeung
Distance-based methods in pattern recognition and machine learning have to rely on a similarity or dissimilarity measure between patterns in the input space. For many applications, Euclidean distance...
Incremental Eigen Decomposition (2003)
James Kwok And, James T. Kwok, Haitao Zhao
Eigen decomposition is a central mathematical tool in many pattern recognition and machine learning techniques. However, it becomes computationally infeasible in the presence of a large set of...
In using the ffl-support vector regression (ffl-SVR) algorithm, one has to decide a suitable value for the insensitivity parameter ffl. Smola et al. considered its "optimal" choice by...
The pre-image problem in kernel methods (2003)
In this paper, we address the problem of finding the pre-image of a feature vector in the feature space induced by a kernel. This is of central importance in some kernel applications, such as on...
Distance Metric Learning with Kernels (2003)
Ivor W. Tsang, James T. Kwok, Clear Water Bay
In this paper, we propose a feature weighting method that works in both the input space and the kernel-induced feature space. It assumes only the availability of similarity (dissimilarity)...
Eigenvoice Speaker Adaptation via Composite Kernel PCA (2003)
James T. Kwok, Brian Mak, Simon Ho
Eigenvoice speaker adaptation has been shown to be effective when only a small amount of adaptation data is available. At the heart of the method is principal component analysis (PCA) employed to...
Fusing Images with Multiple Focuses using (2002)
Support Vector Machines, Shutao Li, James T. Kwok, Yaonan Wang
Optical lenses, particularly those with long focal lengths, suffer from the problem of limited depth of field. Consequently, it is often difficult to obtain good focus for all the objects in the...
Bayesian support vector regression (2001)
We show that the Bayesian evidence framework can be applied to both ffl-support vector regression (ffl-SVR) and-support vector regression (-SVR) algorithms. Standard SVR training can be regarded as...
Rival Penalized Competitive Learning for Model-Based Sequence Clustering (2000)
In this paper, we propose a model-based, competitive learning procedure for the clustering of variable-length sequences. Hidden Markov models (HMMs) are used as representations for the cluster...
William K. Cheung, James T. Kwok, Martin H. Law, Kwok-ching Tsui
As Internet commerce becomes more popular, customers' preferences on various products can now be readily acquired on-line via various e-commerce systems. Properly mining this extracted data can...