Lawrence K. Saul

Topics in High Dimensional Data Analysis (2008)

We derive multiplicative updates for solving the nonnegative quadratic programming problem in support vector machines (SVMs). The updates have a simple closed form, and we prove that they converge...

Bharath K. Sriperumbudur, Instructor Prof, Lawrence K. Saul

Eigenvalue problems are rampant in machine learning and statistics and appear in the context of classification, dimensionality reduction, etc. In this paper, we consider a cardinality constrained...

Category: Speech and Signal Processing, Applications Preference: Oral – includes live Karaoke performance Real time voice processing with audiovisual feedback: toward autonomous agents with perfect pitch (2008)

Lawrence K. Saul, Daniel D. Lee, Charles L. Isbell, Yann Lecun

We have implemented a real time front end for detecting voiced speech and estimating its fundamental frequency. The front end performs the signal processing for voice-driven agents that attend to the...

Abstract (2008)

Kilian Q. Weinberger, Qihui Zhu, Fei Sha, Lawrence K. Saul

In many areas of science and engineering, the problem arises how to discover low dimensional representations of high dimensional data. Recently, a number of researchers have converged on common...

LETTER Communicated by Radford Neal Attractor Dynamics in Feedforward Neural Networks (2007)

Robust Detection of Phonetic Features in Critical Bands of Speech (2007)

We study the probabilistic generative models parameterized by feedforward neural networks. An attractor dynamics for probabilistic inference in these models is derived from a mean field approximation...

Lawrence K. Saul, Mazin G. Rahim, Jont B. Allen

We consider how to detect phonetic features in noisy bandlimited speech. We propose an automatic method based on the hypothesis that independent feature detectors, working in parallel, account for...

Representing Periodic Structure in Speech (2007)

Lawrence K. Saul, Jont B. Allen

An eigenvalue method is developed for analyzing periodic structure in speech. Signals are analyzed by a matrix diagonalization reminiscent of methods for principal component analysis (PCA) and...

Learning From Examples In Critical Bands Of Speech (2007)

Lawrence K. Saul, Mazin G. Rahim, Jont B. Allen

We propose to mimic auditory strategies for recognizing noisy or distorted speech. Motivated by psychoacoustic data on human performance, we begin by studying how to detect the phonetic feature of...

Conformal Component Analysis. unpublished notes ( (2007)

Large margin hidden Markov models for automatic speech recognition (2007)

In this note, we give the details of deriving the algorithm for computing Conformal Component Analysis (CCA) [1], with semidefinite programming. To cite this note, use Fei Sha and Lawrence K. Saul,...

Large margin hidden Markov models for automatic speech recognition (2007)

We study the problem of parameter estimation in continuous density hidden Markov models (CD-HMMs) for automatic speech recognition (ASR). As in support vector machines, we propose a learning...

An introduction to nonlinear dimensionality reduction by maximum variance unfolding (2006)

We study the problem of parameter estimation in continuous density hidden Markov models (CD-HMMs) for automatic speech recognition (ASR). As in support vector machines, we propose a learning...

Kilian Q. Weinberger, Lawrence K. Saul

Distance metric learning for large margin nearest neighbor classification (2006)

Kilian Q. Weinberger, John Blitzer, Lawrence K. Saul

We show how to learn a Mahanalobis distance metric for k-nearest neighbor (kNN) classification by semidefinite programming. The metric is trained with the goal that the k-nearest neighbors always...

Distance metric learning for large margin nearest neighbor classification (2006)

Kilian Q. Weinberger, John Blitzer, Lawrence K. Saul

We show how to learn a Mahanalobis distance metric for k-nearest neighbor (kNN) classification by semidefinite programming. The metric is trained with the goal that the k-nearest neighbors always...

Visualization of Low Dimensional Structure in Tonal Pitch Space (2005)

Burgoyne, J. Ashley, Saul, Lawrence K

In his 2001 monograph Tonal Pitch Space, Fred Lerdahl defined a distance function over tonal and post-tonal harmonies distilled from years of research on music cognition. Although this work...

Nonlinear Dimensionality Reduction by Semidefinite Programming and Kernel Matrix Factorization (2005)

Kilian Q. Weinberger, Benjamin D. Packer, Lawrence K. Saul

We describe an algorithm for nonlinear dimensionality reduction based on semidefinite programming and kernel matrix factorization.

Analysis and extension of spectral methods for nonlinear dimensionality reduction (2005)

Real-time pitch determination of one or more voices by nonnegative matrix factorization (2005)

Many unsupervised algorithms for nonlinear dimensionality reduction, such as locally linear embedding (LLE) and Laplacian eigenmaps, are derived from the spectral decompositions of sparse matrices....

Real-time pitch determination of one or more voices by nonnegative matrix factorization (2005)

An auditory “scene”, composed of overlapping acoustic sources, can be viewed as a complex object whose constituent parts are the individual sources. Pitch is known to be an important cue for...

VISUALIZATION OF LOW DIMENSIONAL STRUCTURE IN TONAL PITCH SPACE (2005)

An auditory “scene”, composed of overlapping acoustic sources, can be viewed as a complex object whose constituent parts are the individual sources. Pitch is known to be an important cue for...

J. Ashley Burgoyne, Lawrence K. Saul

Real-Time Pitch Determination of One or More Voices by Nonnegative Matrix Factorization (2004)

Sha, Fei, Saul, Lawrence K

An auditory "scene", composed of overlapping acoustic sources, can be viewed as a complex object whose constituent parts are the individual sources. Pitch is known to be an important cue for auditory...

Hierarchical Distributed Representations for Statistical Language Modeling (2004)

Blitzer, John, Weinberger, Kilian Q, Saul, Lawrence K, Pereira, Fernando C.N.

Statistical language models estimate the probability of a word occurring in a given context. The most common language models rely on a discrete enumeration of predictive contexts (e.g., n-grams) and...

Modeling Distances in Large-Scale Networks by Matrix Factorization (2004)

Mao, Yun, Saul, Lawrence K

In this paper, we propose a model for representing and predicting distances in large-scale networks by matrix factorization. The model is useful for network distance sensitive applications, such as...

Learning a kernel matrix for nonlinear dimensionality reduction (2004)

Weinberger, Kilian Q, Sha, Fei, Saul, Lawrence K

We investigate how to learn a kernel matrix for high dimensional data that lies on or near a low dimensional manifold. Noting that the kernel matrix implicitly maps the data into a nonlinear feature...

Unsupervised learning of image manifolds by semidefinite programming (2004)

Weinberger, Kilian Q, Saul, Lawrence K

Can we detect low dimensional structure in high dimensional data sets of images and video? The problem of dimensionality reduction arises often in computer vision and pattern recognition. In this...

Exploratory analysis and visualization of speech and music by locally linear embedding (2004)

Jain, Viren, Saul, Lawrence K

Many problems in voice recognition and audio processing involve feature extraction from raw waveforms. The goal of feature extraction is to reduce the dimensionality of the audio signal while...

Multiband statistical learning for f0 estimation in speech (2004)

Sha, Fei, Burgoyne, J. Ashley, Saul, Lawrence K

We investigate a simple algorithm that combines multiband processing and least squares fits to estimate f0 contours in speech. The algorithm is untraditional in several respects: it makes no use of...

Nonnegative deconvolution for time of arrival estimation (2004)

Lin, Yuanqing, Lee, Daniel D, Saul, Lawrence K

The interaural time difference (ITD) of arrival is a primary cue for acoustic sound source localization. Traditional estimation techniques for ITD based upon cross-correlation are related to...

Hierarchical distributed representations for statistical language modeling (2004)

John Blitzer, Kilian Q. Weinberger, Lawrence K. Saul

Statistical language models estimate the probability of a word occurring in a given context. The most common language models rely on a discrete enumeration of predictive contexts (e.g., n-grams) and...

Nonnegative deconvolution for time of arrival estimation (2004)

Yuanqing Lin, Daniel D. Lee, Lawrence K. Saul

The interaural time difference (ITD) of arrival is a primary cue for acoustic sound source localization. Traditional estimation techniques for ITD based upon cross-correlation are related to...

Learning a kernel matrix for nonlinear dimensionality reduction (2004)

Kilian Q. Weinberger, Fei Sha, Lawrence K. Saul

We investigate how to learn a kernel matrix for high dimensional data that lies on or near a low dimensional manifold. Noting that the kernel matrix implicitly maps the data into a nonlinear feature...

Learning a kernel matrix for nonlinear dimensionality reduction (2004)

Kilian Q. Weinberger, Fei Sha, Lawrence K. Saul

We investigate how to learn a kernel matrix for high dimensional data that lies on or near a low dimensional manifold. Noting that the kernel matrix implicitly maps the data into a nonlinear feature...

Statistical signal processing with nonnegativity constraints (2003)

Saul, Lawrence K, Sha, Fei, Lee, Daniel D

Nonnegativity constraints arise frequently in statistical learning and pattern recognition. Multiplicative updates provide natural solutions to optimizations involving these constraints. One well...

Multiplicative Updates for Large Margin Classifiers (2003)

Sha, Fei, Saul, Lawrence K, Lee, Daniel D

Various problems in nonnegative quadratic programming arise in the training of large margin classifiers. We derive multiplicative updates for these problems that converge monotonically to the desired...

Learning High Dimensional Correspondences from Low Dimensional Manifolds (2003)

Ham, Ji Hun, Lee, Daniel D, Saul, Lawrence K

Many different high dimensional data sets are characterized by the same underlying modes of variability. When these modes of variability are continuous and few in number, they can be viewed as...

Think Globally, Fit Locally : Unsupervised Learning of Low Dimensional Manifolds (2003)

Saul, Lawrence K, Roweis, Sam T

The problem of dimensionality reduction arises in many fields of information processing, including machine learning, data compression, scientific visualization, pattern recognition, and neural...

Multiplicative updates for large margin classifiers (2003)

Learning high dimensional correspondences from low dimensional manifolds (2003)

Abstract. Various problems in nonnegative quadratic programming arise in the training of large margin classifiers. We derive multiplicative updates for these problems that converge monotonically to...

Ji Hun Ham, Daniel D. Lee, Lawrence K. Saul

Many different high dimensional data sets are characterized by the same underlying modes of variability. When these modes of variability are continuous and few in number, they can be viewed as...

Learning high dimensional correspondences from low dimensional manifolds (2003)

Ji Hun Ham, Daniel D. Lee, Lawrence K. Saul

Many different high dimensional data sets are characterized by the same underlying modes of variability. When these modes of variability are continuous and few in number, they can be viewed as...

Multiplicative updates for large margin classifiers (2003)

Think globally, fit locally: unsupervised learning of low dimensional manifolds (2003)

Abstract. Various problems in nonnegative quadratic programming arise in the training of large margin classifiers. We derive multiplicative updates for these problems that converge monotonically to...

Lawrence K. Saul, Sam T. Roweis, Yoram Singer

The problem of dimensionality reduction arises in many fields of information processing, including machine learning, data compression, scientific visualization, pattern recognition, and neural...

Multiplicative updates for nonnegative quadratic programming in support vector machines (2003)

Think globally, fit locally: unsupervised learning of low dimensional manifolds (2003)

support vector machines

A generalized linear model for principal component analysis of binary data (2003)

The problem of dimensionality reduction arises in many fields of information processing, including machine learning, data compression, scientific visualization, pattern recognition, and neural...

Andrew I. Schein, Lawrence K. Saul, Lyle H. Ungar

We investigate a generalized linear model for dimensionality reduction of binary data. The model is related to principal component analysis (PCA) in the same way that logistic regression is related...

A Generalized Linear Model for Principal Component Analysis of Binary Data (2003)

Andrew I. Schein, Andrew I. Lawrence, Lawrence K. Saul, Lyle H. Ungar

VVe investigate a generalized linear model fbr dimensionality reduction of binary data. The model is related to principal component anal- ysis (PCA) in the same way that logistic regression is...

LeCun Real time voice processing with audiovisual feedback: toward autonomous agents with perfect pitch (2003)

Lawrence K. Saul, Daniel D. Lee, Charles L. Isbell, Yann Lecun

We have implemented a real time front end for detecting voiced speech and estimating its fundamental frequency. The front end performs the signal processing for voice-driven agents that attend to the...

Multiplicative updates for nonnegative quadratic programming in support vector machines (2003)

Think globally, fit locally: unsupervised learning of low dimensional manifolds (2003)

We derive multiplicative updates for solving the nonnegative quadratic programming problem in support vector machines (SVMs). The updates have a simple closed form, and we prove that they converge...

Lawrence K. Saul, Yoram Singer

The problem of dimensionality reduction arises in many fields of information processing, including machine learning, data compression, scientific visualization, pattern recognition, and neural...

A generalized linear model for principal component analysis of binary data (2003)

Andrew I. Schein, Lawrence K. Saul, Lyle H. Ungar

We investigate a generalized linear model for dimensionality reduction of binary data. The model is related to principal component analysis (PCA) in the same way that logistic regression is related...

Think globally, fit locally: unsupervised learning of low dimensional manifolds (2003)

Multiplicative updates for nonnegative quadratic programming in support vector machines (2002)

The problem of dimensionality reduction arises in many fields of information processing, including machine learning, data compression, scientific visualization, pattern recognition, and neural...

Sha, Fei, Saul, Lawrence K, Lee, Daniel D

We derive multiplicative updates for solving the nonnegative quadratic programming problem in support vector machines (SVMs). The updates have a simple closed form, and we prove that they converge...

Real time voice processing with audiovisual feedback : toward autonomous agents with perfect pitch (2002)

Saul, Lawrence K, Lee, Daniel D, Isbell, Charles L, LeCun, Yann

We have implemented a real time front end for detecting voiced speech and estimating its fundamental frequency. The front end performs the signal processing for voice-driven agents that attend to the...

Multiplicative updates for classification by mixture models (2002)

Lawrence K. Saul, Daniel D. Lee

We investigate a learning algorithm for the classification of nonnegative data by mixture models. Multiplicative update rules are derived that directly optimize the performance of these models as...

Real time voice processing with audiovisual feedback: toward autonomous agents with perfect pitch (2002)

Lawrence K. Saul, Daniel D. Lee, Charles L. Isbell, Yann Lecun

We have implemented a real time front end for detecting voiced speech and estimating its fundamental frequency. The front end performs the signal processing for voice-driven agents that attend to the...

Multiplicative updates for classification by mixture models (2002)

Lawrence K. Saul, Daniel D. Lee

We investigate a learning algorithm for the classification of nonnegative data by mixture models. Multiplicative update rules are derived that directly optimize the performance of these models as...

Global coordination of local linear models (2002)

Sam Roweis, Lawrence K. Saul, Geoffrey E. Hinton

High dimensional data that lies on or near a low dimensional manifold can be described by a collection of local linear models. Such a description, however, does not provide a global parameterization...

Global Coordination of Local Linear Models (2001)

Roweis, Sam, Saul, Lawrence K, Hinton, Geoffrey E

High dimensional data that lies on or near a low dimensional manifold can be described by a collection of local linear models. Such a description, however, does not provide a global parameterization...

Multiplicative Updates for Classification by Mixture Models (2001)

Saul, Lawrence K, Lee, Daniel D

We investigate a learning algorithm for the classification of nonnegative data by mixture models. Multiplicative update rules are derived that directly optimize the performance of these models as...

Periodic component analysis: an eigenvalue method for representing periodic structure in speech (2001)

Lawrence K. Saul, Jont B. Allen

An eigenvalue method is developed for analyzing periodic structure in speech. Signals are analyzed by a matrix diagonalization reminiscent of methods for principal component analysis (PCA) and...

Periodic component analysis: an eigenvalue method for representing periodic structure in speech (2001)

Lawrence K. Saul, Jont B. Allen

An eigenvalue method is developed for analyzing periodic structure in speech. Signals are analyzed by a matrix diagonalization reminiscent of methods for principal component analysis (PCA) and...

An Introduction to Locally Linear Embedding (2000)

Markov processes on curves (2000)

Lawrence K. Saul, Mazin G. Rahim

1 Introduction The automatic segmentation of continuous trajectories poses a challenging problem in machine learning. The problem arises whenever a multidimensional trajectory fx(t)jt 2 [0; o /]g...

Markov Processes on Curves (2000)

Lawrence K. Saul, Mazin G. Rahim

We study the classification problem that arises when two variables--- one continuous (x), one discrete (s)---evolve jointly in time. We suppose that the vector x traces out a smooth multidimensional...

An Introduction to Locally Linear Embedding (2000)

Mixed memory markov models: Decomposing complex stochastic processes as mixtures of simpler ones (1999)

Lawrence K. Saul, Michael I. Jordan, Padhraic Smyth

Abstract. We study Markov models whose state spaces arise from the Cartesian product of two or more discrete random variables. We show how to parameterize the transition matrices of these models as a...

Fast Learning by Bounding Likelihoods in Sigmoid Type Belief Networks. (1998)

Jaakkola, Tommi S., Saul, Lawrence K., Jordan, Michael I.

Sigmoid type belief networks, a class of probabilistic neural networks, provide a natural framework for compactly representing probabilistic information in a variety of unsupervised and supervised...

An Introduction to Variational Methods for Graphical Methods (1998)

Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola, Lawrence K. Saul

. This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of...

Mixed memory Markov models: decomposing complex stochastic processes as mixtures of simpler ones (1998)

Hidden Markov decision trees (1997)

. We study Markov models whose state spaces arise from the Cartesian product of two or more discrete random variables. We show how to parameterize the transition matrices of these models as a convex...

Michael I. Jordan, Zoubin Ghahramani, Lawrence K. Saul

We study a time series model that can be viewed as a decision tree with Markov temporal structure. The model is intractable for exact calculations, thus we utilize variational approximations. We...

Hidden Markov decision trees (1997)

Michael Jordan, Zoubin Ghahramani, Lawrence K. Saul

We study a time series model that can be viewed as a decision tree with Markov temporal structure. The model is intractable for exact calculations, thus we utilize variational approximations. We...

Mean Field Theory for Sigmoid Belief Networks (1996)

Saul, Lawrence K., Jaakkola, Tommi, Jordan, Michael I.

We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in...

Mean Field Theory for Sigmoid Belief Networks (1996)

Saul, Lawrence K., Jaakkola, Tommi, Jordan, Michael I.

We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in...

Fast Learning by Bounding Likelihoods in Sigmoid Type Belief Networks (1996)

Jaakkola, Tommi S., Saul, Lawrence K., Jordan, Michael I.

Sigmoid type belief networks, a class of probabilistic neural networks, provide a natural framework for compactly representing probabilistic information in a variety of unsupervised and supervised...

Fast Learning by Bounding Likelihoods in Sigmoid Type Belief Networks (1996)

Jaakkola, Tommi S., Saul, Lawrence K., Jordan, Michael I.

Sigmoid type belief networks, a class of probabilistic neural networks, provide a natural framework for compactly representing probabilistic information in a variety of unsupervised and supervised...

Fast learning by bounding likelihoods in sigmoid type belief networks (1996)

Tommi S. Jaakkola, Lawrence K. Saul, Michael I. Jordan

This publication can be retrieved by anonymous ftp to publications.ai.mit.edu. Sigmoid type belief networks, a class of probabilistic neural networks, provide a natural framework for compactly...

Mean field theory for sigmoid belief networks (1996)

Lawrence K. Saul, Tommi Jaakkola, Michael I. Jordan

This publication can be retrieved by anonymous ftp to publications.ai.mit.edu. We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field...

Learning curve bounds for markov decision processes with undiscounted rewards (1996)

Lawrence K. Saul, Satinder P. Singh

Markov decision processes (MDPs) with undiscounted rewards represent an important class of problems in decision and control. The goal of learning in these MDPs is to find a policy that yields the...

Fast learning by bounding likelihoods in sigmoid type belief networks (1996)

Tommi Jaakkola, Lawrence K. Saul, Michael I. Jordan

Sigmoid type belief networks, a class of probabilistic neural networks, provide a natural framework for compactly representing probabilistic information in a variety of unsupervised and supervised...

Mean field theory for sigmoid belief networks (1996)

Lawrence K. Saul, Tommi Jaakkola, Michael I. Jordan

We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in...

Fast Learning by Bounding Likelihoods in Sigmoid Type Belief Networks (1996)

Tommi S. Jaakkola, Lawrence K. Saul, Michael I. Jordan

Sigmoid type belief networks, a class of probabilistic neural networks, provide a natural framework for compactly representing probabilistic information in a variety of unsupervised and supervised...

Mean Field Theory for Sigmoid Belief Networks (1996)

Lawrence K. Saul, Tommi Jaakkola, Michael I. Jordan

We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in...

Mean Field Theory for Sigmoid Belief Networks (1996)

Lawrence K. Saul, Tommi Jaakkola, Michael I. Jordan

We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in...

Mean field theory for sigmoid belief networks (1996)

Lawrence K. Saul, Tommi Jaakkola, Michael I. Jordan

We develop a mean eld theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean eld theory provides a tractable approximation to the true probability distribution in...

Markov Decision Processes in Large State Spaces (1995)

Lawrence K. Saul, Satinder P. Singh

In this paper we propose a new framework for studying Markov decision processes (MDPs), based on ideas from statistical mechanics. The goal of learning in MDPs is to find a policy that yields the...

Boltzmann Chains and Hidden Markov Models (1995)

Attractor Dynamics in Feedforward Neural Networks

We propose a statistical mechanical framework for the modeling of discrete time series. Maximum likelihood estimation is done via Boltzmann learning in one-dimensional networks with tied weights. We...