Robert C. Williamson

Generalised Pinsker Inequalities (2009)

Reid, Mark D., Williamson, Robert C.

We generalise the classical Pinsker inequality which relates variational divergence to Kullback-Liebler divergence in two ways: we consider arbitrary f-divergences in place of KL divergence, and we...

Information, Divergence and Risk for Binary Experiments (2009)

Reid, Mark D., Williamson, Robert C.

We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically...

Learnability of Probabilistic Automata via Oracles (2008)

Omri Guttman, Robert C. Williamson

Abstract. Efficient learnability using the state merging algorithm is known for a subclass of probabilistic automata termed µ-distinguishable. In this paper, we prove that state merging algorithms...

The Australian National University (2008)

Online Bayes, Point Machines, Edward Harrington, Ralf Herbrich, Jyrki Kivinen, John C. Platt, ...

Abstract. We present a new and simple algorithm for learning large margin classifiers that works in a truly online manner. The algorithm generates a linear classifier by averaging the weights...

Learnability of Probabilistic Automata via Oracles (2008)

Omri Guttman, Robert C. Williamson

Abstract. Efficient learnability using the state merging algorithm is known for a subclass of probabilistic automata termed µ-distinguishable. In this paper, we prove that state merging algorithms...

Abstract (2008)

Alex J. Smola, Bernhard Schölkopf, John Shawe-taylor, Robert C. Williamson

Effective methods of capacity control via uniform convergence bounds for function expansions have been largely limited to Support Vector machines, where good bounds are obtainable by the entropy...

LETTER Communicated by John Platt New Support Vector Algorithms (2008)

Bernhard Schölkopf, Alex J. Smola, Robert C. Williamson, Peter L. Bartlett

We propose a new class of support vector algorithms for regression and classi�cation. In these algorithms, a parameter º lets one effectively control the number of support vectors. While this can...

Abstract (2008)

Thore Graepel, Ralf Herbrich, Robert C. Williamson

We present an improvement ofNoviko 's perceptron convergence theorem. Reinterpreting this mistake bound as a margin dependent sparsity guarantee allows us to give aPAC{style generalisation error...

Learnability of Probabilistic Automata via Oracles (2008)

Omri Guttman, Robert C. Williamson

Abstract. Efficient learnability using the state merging algorithm is known for a subclass of probabilistic automata termed µ-distinguishable. In this paper, we prove that state merging algorithms...

ARTICLE Communicated by Vladimir Vapnik Estimating the Support of a High-Dimensional Distribution (2008)

Bernhard Schölkopf, John C. Platt, John Shawe-taylor, Alex J. Smola, Robert C. Williamson

Suppose you are given some data set drawn from an underlying probability distribution P and you want to estimate a “simple ” subset S of input space such that the probability that a test point...

and Australian National University (2008)

Cheng Soon Ong, Alexander J. Smola, Robert C. Williamson, Ralf Herbrich

This paper addresses the problem of choosing a kernel suitable for estimation with a support vector machine, hence further automating machine learning. This goal is achieved by defining a reproducing...

Removing Loops from LDPC Codes (2008)

James A. Mcgowan, Robert C. Williamson

Abstract — Small loops in the graphical representations of lowdensity parity-check codes are hypothesised to be detrimental to the code’s performance. In this paper we demonstrate a procedure for...

Learning the Kernel with Hyperkernels Cheng Soon Ong (2008)

Alexander J. Smola, Robert C. Williamson

Editor: U.N. Known (joint publication with www.kernel-machines.org) This paper addresses the problem of choosing a kernel suitable for estimation with a Support Vector Machine, hence further...

CHANNEL EQUALIZATION AND THE BAYES POINT MACHINE (2008)

Edward Harrington, Jyrki Kivinen, Robert C. Williamson

Equalizers trained with a large margin have an ability to better handle noise in unseen data and drift in the target solution. We present a method of approximating the Bayes optimal strategy which...

National ICT Australia Technical Reports ISSN 1833-9646 Posterior Cramér-Rao Bound for Acoustic Source Tracking in Reverberant Environments (2008)

Eric A. Lehmann, Robert C. Williamson

The research presented in this paper is motivated by the desire to determine a lower bound on the estimation error for acoustic source localisation and tracking (ASLT) algorithms operating in...

Particle Filter Design Using Importance Sampling for Acoustic Source Localisation and Tracking in Reverberant Environments (2008)

Eric A. Lehmann, Robert C. Williamson

Sequential Monte Carlo methods have been recently proposed to deal with the problem of acoustic source localisation and tracking using an array of microphones. Previous implementations make use of...

The Entropy Regularization Information Criterion (2007)

Alex J. Smola, John Shawe-taylor, Bernhard Schölkopf, Bernhard Sch Olkopf, Robert C. Williamson

Effective methods of capacity control via uniform convergence bounds for function expansions have been largely limited to Support Vector machines, where good bounds are obtainable by the entropy...

An Analysis Of The Exponentiated Gradient Descent Algorithm (2007)

Simon I. Hill, Robert C. Williamson

This paper analyses three algorithms recently studied in the Computational Learning Theory community: the Gradient Descent (GD) Algorithm, the Exponentiated Gradient Algorithm with Positive and...

Sample Complexity versus Approximation Error (2007)

Peter Bartlett, Robert C. Williamson

We consider the problem of learning an unknown real-valued function from a sequence of values of the function at randomly chosen points when the estimates are constrained to some class of functions...

Lower Bounds on the VC-Dimension of Smoothly Parametrized Function Classes (2007)

Wee Sun, Lee Peter, L. Bartlett, Robert C. Williamson

We examine the relationship between the VC-dimension and the number of parameters of a smoothly parametrized function class. We show that the VCdimension of such a function class is at least k if...

1 (2007)

Edward Harrington, Ralf Herbrich, Jyrki Kivinen, John C. Platt, Robert C. Williamson

Abstract. We present a new and simple algorithm for learning large margin classiers that works in a truly online manner. The algorithm generates a linear classier by averaging the weights associated...

Spatial Aliasing for Nearfield Sensor Arrays (2007)

Thushara D. Abhayapala, Rodney A. Kennedy, Robert C. Williamson

This paper investigates the presence of spatial aliasing due to operating a linear array in the nearfield. It shows that the standard half wavelength sensor spacings rule, which guarantees no...

Australian National University (2007)

Thushara D. Abhayapala, Rodney A. Kennedy, Robert C. Williamson, Darren B. Ward

A nearfield broadband adaptive beamforming approach based on the modal expansion of the solution to the spherical wave equation is formulated. It provides an efficient parameterization for the...

FARFIELD ARRAY WEIGHT REDESIGN FOR NEARFIELD BEAMFORMING (2007)

Thushara D. Abhayapala, Rodney A. Kennedy, Robert C. Williamson

This paper presents a new method for nearfield linear array beamforming that achieves a desired beampattern (as a function of direction) at any nominal finite distance from the array origin. Given a...

Agnostic Learning Nonconvex Function Classes (2007)

Shahar Mendelson And, Shahar Mendelson, Robert C. Williamson

We consider the sample complexity of agnostic learning with respect to squared loss. It is known that if the function class F used for learning is convex then one can obtain better sample complexity...

LETTER Communicated by John Platt New Support Vector Algorithms (2007)

Bernhard Schölkopf, Alex J. Smola, Robert C. Williamson, Peter L. Bartlett

We propose a new class of support vector algorithms for regression and classification. In these algorithms, a parameter ν lets one effectively control the number of support vectors. While this can...

Hyperkernels (2007)

Cheng Soon Ong, Alexander J. Smola, Er J. Smola, Robert C. Williamson

We consider the problem of choosing a kernel suitable for estimation using a Gaussian Process estimator or a Support Vector Machine. A novel solution is presented which involves defining a...

Particle Filter Design Using Importance Sampling for Acoustic Source Localisation and Tracking in Reverberant Environments (2006)

Eric A. Lehmann, Robert C. Williamson

Sequential Monte Carlo methods have been recently proposed to deal with the problem of acoustic source localisation and tracking using an array of microphones. Previous implementations make use of...

Importance Sampling Particle Filter for Robust Acoustic Source Localisation and Tracking in Reverberant Environments (2005)

Eric A. Lehmann, Robert C. Williamson

means of a sample-based representation of the posterior density, and it is able to deal with nonlinear and non-Gaussian problems. Because a PF delivers location estimates based on a series of past...

Learning the kernel with hyperkernels (2005)

Cheng Soon Ong, Alexander J. Smola, Robert C. Williamson, Ralf Herbrich

This paper addresses the problem of choosing a kernel suitable for estimation with a Support Vector Machine, hence further automating machine learning. This goal is achieved by defining a Reproducing...

Online Learning with Kernels (2003)

Jyrki Kivinen, Alexander J. Smola, Robert C. Williamson

Kernel based algorithms such as support vector machines have achieved considerable success in various problems in the batch setting where all of the training data is available in advance. Support...

Learning the Kernel with Hyperkernels (2003)

Cheng Soon Ong, Alexander J. Smola, Robert C. Williamson

This paper addresses the problem of choosing a kernel suitable for estimation with a Support Vector Machine, hence further automating machine learning. This goal is achieved by defining a Reproducing...

Particle Filtering Algorithms for Tracking an Acoustic Source in a Reverberant Environment (2003)

Darren Ward Member, Darren B. Ward, Eric A. Lehmann, Student Member, Robert C. Williamson

Traditional acoustic source localization algorithms attempt to find the current location of the acoustic source using data collected at an array of sensors at the current time only. In the presence...

Experimental Comparison Of Particle Filtering Algorithms (2003)

For Acoustic Source, Eric A. Lehmann, Darren B. Ward, Robert C. Williamson

Traditional acoustic source localization techniques attempt to determine the current location of an acoustic source from data obtained at an array of sensors during the current time only. Recently,...

Hyperkernels (2003)

Cheng Soon Ong, Er J. Smola, Robert C. Williamson

We consider the problem of choosing a kernel suitable for estimation using a Gaussian Process estimator or a Support Vector Machine. A novel solution is presented which involves defining a...

Logic, Trees and Kernels (2003)

Adam Kowalczyk, Alexander J. Smola, Robert C. Williamson

Kernel based methods achieved much of their initial success on problems with real valued attributes. There are many problems with discrete attributes (including Boolean) and in this paper we present...

Algorithmic luckiness (2002)

Ralf Herbrich, Robert C. Williamson

Classical statistical learning theory studies the generalisation performance of machine learning algorithms rather indirectly. One of the main detours is that algorithms are studied in terms of the...

Large margin classification for moving targets (2002)

Jyrki Kivinen, Alex J. Smola, Robert C. Williamson

Abstract. We consider using online large margin classification algorithms in a setting where the target classifier may change over time. The algorithms we consider are Gentile’s Alma, and an...

Kernel machines and boolean functions (2002)

Adam Kowalczyk, Alex J. Smola, Robert C. Williamson

We give results about the learnability and required complexity of logical formulae to solve classification problems. These results are obtained by linking propositional logic with kernel machines. In...

Algorithmic luckiness (2002)

Ralf Herbrich, Cb Fb Cambridge, Robert C. Williamson

In contrast to standard statistical learning theory which studies uniform bounds on the expected error we present a framework that exploits the specic learning algorithm used. Motivated by the...

Learning with Kernels (2002)

Jyrki Kivinen, Alex J. Smola, Robert C. Williamson

We consider online learning in a Reproducing Kernel Hilbert Space. Our method is computationally efficient and leads to simple algorithms. In particular we derive update equations for classification,...

The structure of version space (2002)

Ralf Herbrich, Thore Graepel, Robert C. Williamson

We investigate the generalisation performance of consistent classiers, i.e. classiers that are contained in the so-called version space, both from a theoretical and experimental angle. In contrast to...

Exploiting Sparsity in Adaptive Filters (2002)

Richard K. Martin, William A. Sethares, Robert C. Williamson, C. Richard Johnson

This paper studies a class of algorithms called natural gradient (NG) algorithms. The least mean square (LMS) algorithm is derived within the NG framework, and a family of LMS variants that exploit...

Learning with Kernels (2002)

Jyrki Kivinen, Alex J. Smola, Robert C. Williamson

We consider online learning in a Reproducing Kernel Hilbert Space. Our method is computationally efficient and leads to simple algorithms. In particular we derive update equations for classification,...

Kernel machines and boolean functions (2002)

Adam Kowalczyk, Alex J. Smola, Robert C. Williamson

We give results about the learnability and required complexity of logical formulae to solve classification problems. These results are obtained by linking propositional logic with kernel machines. In...

Algorithmic luckiness (2002)

Ralf Herbrich, Cb Fb Cambridge, Robert C. Williamson, A. Cohn

Classical statistical learning theory studies the generalisation performance of machine learning algorithms rather indirectly. One of the main detours is that algorithms are studied in terms of the...

Online Learning with Kernels (2002)

Jyrki Kivinen, Alex J. Smola, Robert C. Williamson

We consider online learning in a Reproducing Kernel Hilbert Space. Our method is computationally efficient and leads to simple algorithms. In particular we derive update equations for classification,...

Algorithmic luckiness (2001)

Ralf Herbrich, Robert C. Williamson

In contrast to standard statistical learning theory which studies uniform bounds on the expected error we present a framework that exploits the specific learning algorithm used. Motivated by the...

The Structure of Version Space (2001)

Ralf Herbrich, Thore Graepel, Robert C. Williamson

We investigate the generalisation performance of consistent classifiers, i.e. classifiers that are contained in the so-called version space, both from a theoretical and experimental angle. In...

A generalized representer theorem (2001)

Bernhard Schölkopf, Ralf Herbrich, Alex J. Smola, Robert C. Williamson

Wahba's classical representer theorem states that the solutions of certain risk minimization problems involving an empirical risk term and a quadratic regularizer can be written as expansions in...

Regularized Principal Manifolds (2001)

Alexander J. Smola, Sebastian Mika, Bernhard Schölkopf, Robert C. Williamson

Many settings of unsupervised learning can be viewed as quantization problems - the minimization of the expected quantization error subject to some restrictions. This allows the use of tools such as...

Prior Knowledge and Preferential Structures in Gradient Descent Learning Algorithms (2001)

Robert E. Mahony, Robert C. Williamson

A family of gradient descent algorithms for learning linear functions in an online setting is considered. The family includes the classical LMS algorithm as well as new variants such as the...

Estimating the Support of a High-Dimensional Distribution (2001)

Bernhard Schölkopf, John C. Platt, John Shawe-taylor, Alex J. Smola, Robert C. Williamson

This article describes an algorithm that finds regions close to C(#). Our class is defined implicitly via a kernel k as the set of half-spaces in an SV feature space. We do not try to minimize the...

Convergence of Exponentiated Gradient Algorithms (2001)

Simon Hill And, Simon I. Hill, Robert C. Williamson

This paper studies three related algorithms: the (traditional) Gradient Descent (GD) Algorithm, the Exponentiated Gradient Algorithm with Positive and Negative weights (EG algorithm) and the...

Prior knowledge and preferential structures in gradient descent algorithms (2001)

Robert E. Mahony, Robert C. Williamson

A family of gradient descent algorithms for learning linear functions in an online setting is considered. The family includes the classical LMS algorithm as well as new variants such as the...

Prior knowledge and preferential structures in gradient descent algorithms (2001)

Robert E. Mahony, Robert C. Williamson

A family of gradient descent algorithms for learning linear functions in an online setting is considered. The family includes the classical LMS algorithm as well as new variants such as the...

Exact simplification of support vector solutions (2001)

Tom Downs, Kevin E Gates, Annette Masters, Nello Cristianini, John Shaw-taylor, Robert C. Williamson, ...

This paper demonstrates that standard algorithms for training support vector machines generally produce solutions with a greater number of support vectors than are strictly necessary. An algorithm is...

Regularization with dot-product kernels (2000)

Alex J. Smola, Robert C. Williamson

In this paper we give necessary and sucient conditions under which kernels of dot product type k(x; y) = k(x y) satisfy Mercer 's condition and thus may be used in Support Vector Machines (SVM),...

Entropy Numbers of Linear Function Classes (2000)

Robert C. Williamson, Alex J. Smola, Bernhard Schölkopf, Bernhard Sch Olkopf

This paper collects together a miscellany of results originally motivated by the analysis of the generalization performance of the "maximum-margin" algorithm due to Vapnik and others. The...

Riemannian structure of some new gradient descent learning algorithms (2000)

Robert E. Mahony, Robert C. Williamson

We consider some generalizations of the classical LMS learning algorithm including the Exponentiated Gradient (EG) algorithm. We show how one can develop these algorithms in terms of a prior...

Entropy numbers of linear function classes (2000)

Robert C. Williamson

This paper collects together a miscellany of results originally motivated by the analysis of the generalization performance of the “maximum-margin ” algorithm due to Vapnik and others. The key...

Regularization with dot-product kernels (2000)

Alex J. Smola, Zoltán L. Óvári, Robert C. Williamson

In this paper we give necessary and sufficient conditions under which kernels of dot product type k(x, y) = k(x · y) satisfy Mercer’s condition and thus may be used in Support Vector Machines...

Imposing Pattern Nulls on Broadband Array Responses (1999)

Kootsookos, Peter J., Ward, Darren B., Williamson, Robert C.

This paper considers the problem of altering a quiescent design for an array of omni-directional sensors so that the altered design rejects a farfield broadband signal from a given direction. This...

Covering Numbers for Support Vector Machines (1999)

Guo, Ying, Bartlett, Peter L., Shawe-Taylor, John, Williamson, Robert C.

Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization...

Covering Numbers for Support Vector Machines (1999)

Guo, Ying, Bartlett, Peter L., Shawe-Taylor, John, Williamson, Robert C.

Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization...

Covering Numbers for Support Vector Machines (1999)

Guo, Ying, Bartlett, Peter L., Shawe-Taylor, John, Williamson, Robert C.

Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization...

Imposing Pattern Nulls on Broadband Array Responses (1999)

Kootsookos, Peter J., Ward, Darren B., Williamson, Robert C.

This paper considers the problem of altering a quiescent design for an array of omni-directional sensors so that the altered design rejects a farfield broadband signal from a given direction. This...

Imposing Pattern Nulls on Broadband Array Responses (1999)

Kootsookos, Peter J., Ward, Darren B., Williamson, Robert C.

This paper considers the problem of altering a quiescent design for an array of omni-directional sensors so that the altered design rejects a farfield broadband signal from a given direction. This...

Noise modeling for nearfield array optimization (1999)

Thushara D. Abhayapala, Student Member, Rodney A. Kennedy, Robert C. Williamson

Abstract — In this letter, an exact series representation for a nearfield spherically isotropic noise model is introduced. The proposed noise model can be utilized effectively to apply...

Regularized principal manifolds (1999)

Alexander J. Smola, Robert C. Williamson

Editor: Douglas H. Fisher (joint publication with www.kernel-machines.org) Many settings of unsupervised learning can be viewed as quantization problems- the minimization of the expected quantization...

Regularized principal manifolds (1999)

Alex J. Smola, Robert C. Williamson, Sebastian Mika, Bernhard Schölkopf

Abstract. Many settings of unsupervised learning can be viewed as quantization problems — the minimization of the expected quantization error subject to some restrictions. This allows the use of...

Estimating the Support of a High-Dimensional Distribution (1999)

Bernhard Schölkopf, John C. Platt, John Shawe-taylor, Alex J. Smola, Robert C. Williamson

Suppose you are given some dataset drawn from an underlying probability distribution P and you want to estimate a "simple" subset S of input space such that the probability that a test...

Estimating the Support of a High-Dimensional Distribution (1999)

Bernhard Schölkopf, John C. Platt, John Shawe-Taylor, Alex J. Smola, Robert C. Williamson

Suppose you are given some dataset drawn from an underlying probability distribution P and you want to estimate a "simple" subset S of input space such that the probability that a test...

Kernel-Dependent Support Vector Error Bounds (1999)

Bernhard Schölkopf, John Shawe-taylor, Alex J. Smola, Robert C. Williamson

Model selection in Support Vector machines is usually carried out by minimizing the quotient of the radius of the smallest enclosing sphere of the data and the observed margin on the training set. We...

SV Estimation of a Distribution's Support (1999)

Bernhard Schölkopf, Bernhard Sch Olkopf, Robert C. Williamson, Alex Smola, John Shawe-taylor

Suppose you are given some dataset drawn from an underlying probability distribution P and you want to estimate a subset S of input space such that the probability that a test point drawn from P lies...

Generalization Bounds via Eigenvalues of the Gram Matrix (1999)

Bernhard Schölkopf, John Shawe-Taylor, Alexander J. Smola, Bernhard Scholkopf Gmd, Er J. Smola, Robert C. Williamson

Model selection in Support Vector machines is usually carried out by minimizing the quotient of the radius of the smallest enclosing sphere of the data and the observed margin on the training set. We...

Generalization Performance of Regularization Networks and Support Vector Machines via Entropy Numbers of Compact Operators (1999)

Robert C. Williamson, Alex J. Smola, Bernhard Schölkopf

We derive new bounds for the generalization error of kernel machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers. The proofs...

Regularized Principal Manifolds (1999)

Alex J. Smola, Sebastian Mika, Bernhard Schölkopf, Robert C. Williamson

. Many settings of unsupervised learning can be viewed as quantization problems --- the minimization of the expected quantization error subject to some restrictions. This allows the use of tools such...

Noise Modeling for Nearfield Array Optimization (1999)

Thushara Abhayapala Student, Thushara D. Abhayapala, Student Member, Rodney A. Kennedy, Robert C. Williamson

In this letter, an exact series representation for a nearfield spherically isotropic noise model is introduced. The proposed noise model can be utilized effectively to apply wellestablished farfield...

Estimating the Support of a High-Dimensional Distribution (1999)

Bernhard Scholkopf John, John C. Platt, John Shawe-taylor, Alex J. Smola, Robert C. Williamson

Suppose you are given some dataset drawn from an underlying probability distribution and you want to estimate a "simple" subset of input space such that the probability that a test point...

Entropy Numbers, Operators and Support Vector Kernels (1999)

Robert C. Williamson, Alex J. Smola, Bernhard Schölkopf

We derive new bounds for the generalization error of feature space machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers.

Covering numbers for support vector machines (1999)

Ying Guo, Peter L. Bartlett, John Shawe-taylor, Robert C. Williamson

Support vector machines are a type of learning machine related to the maximum margin hyperplane. Until recently, the only bounds on the generalization performance of SV machines (within the PAC...

Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators (1998)

Robert C. Williamson, Alex J. Smola, Bernhard Schölkopf

Abstract—We derive new bounds for the generalization error of kernel machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers....

Entropy Numbers, Operators and Support Vector Kernels (1998)

Robert C. Williamson, Alex J. Smola, Bernhard Schölkopf, Bernhard Scholkopf Gmd

We derive new bounds for the generalization error of feature space machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers. The...

Generalization Bounds for Convex Combinations of Kernel Functions (1998)

Alex J. Smola, Robert C. Williamson, Bernhard Schölkopf, Bernhard Scholkopf Gmd

We derive new bounds on covering numbers for hypothesis classes generated by convex combinations of basis functions. These are useful in bounding the generalization performance of algorithms such as...

Generalization Performance of Regularization Networks and Support Vector Machines via Entropy Numbers of Compact Operators (1998)

Robert C. Williamson, Alex J. Smola, Bernhard Schölkopf, Bernhard Scholkopf Gmd

We derive new bounds for the generalization error of kernel machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers. The proofs...

Structural Risk Minimization over Data-Dependent Hierarchies (1998)

John Shawe-taylor, Peter L. Bartlett, Robert C. Williamson, Martin Anthony

The paper introduces some generalizations of Vapnik's method of structural risk minimisation (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to...

Generalization Performance of Regularization Networks and Support Vector Machines via Entropy Numbers of Compact Operators (1998)

Robert C. Williamson, Alex J. Smola, Bernhard Schölkopf, Bernhard Scholkopf Gmd

We derive new bounds for the generalization error of kernel machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers. The proofs...

Structural risk minimization over data-dependent hierarchies (1998)

John Shawe-taylor, Peter L. Bartlett, Martin Anthony, Robert C. Williamson

The paper introduces some generalizations of Vapnik’s method of structural risk minimisation (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to...

Some results in statistical learning theory with relevance to nonlinear system identification (1998)

Robert C. Williamson

Abstract: Statistical Learning Theory comprises a collection of techniques that have been developed in order to theoretically analyse the performance of neural network and other “learning ”...

The Importance of Convexity in Learning with Squared Loss (1997)

Wee Sun Lee, Peter L. Bartlett, Robert C. Williamson

We show that if the closure of a function class F under the metric induced by some probability distribution is not convex, then the sample complexity for agnostically learning F with squared loss...

Correction to "Lower Bounds on the VC-Dimension of Smoothly Parametrized Function Classes" (1997)

Wee Sun Lee, Peter L. Bartlett, Robert C. Williamson

The paper [3] gives lower bounds on the VC-dimension of various smoothly parametrized function classes. The results were proven by showing a relationship between the uniqueness of decision boundaries...

Correction to "Lower Bounds on the VC-Dimension of Smoothly Parametrized Function Classes" (1997)

Wee Sun, Peter L. Bartlett, Robert C. Williamson

The paper [3] gives lower bounds on the VC-dimension of various smoothly parametrized function classes. The results were proven by showing a relationship between the uniqueness of decision boundaries...

The Importance of Convexity in Learning with Squared Loss (1997)

Wee Sun, Lee Peter, L. Bartlett, Robert C. Williamson

Weshow that if the closure of a function class F under the metric induced by some probability distribution is not convex, then the sample complexity for agnostically learning F with squared loss...

FIR Approximation of Fractional Sample Delay Systems (1996)

Kootsookos, Peter J., Williamson, Robert C.

This paper examines the approximation of fractional delays by FIR systems using various techniques which have previously been reported in the literature. In particular, the equivalence of the...

Frequency Invariant Broadband Beamforming with Exact Null Placement (1996)

Kootsookos, Peter J., Ward, Darren B., Williamson, Robert C.

This paper extends earlier results by Ward, Kennedy and Williamson [1,2] for the design of broadband arrays with frequency-invariant (FI) beam patterns to the case where it is desired to place an...

Frequency Invariant Broadband Beamforming with Exact Null Placement (1996)

Kootsookos, Peter J., Ward, Darren B., Williamson, Robert C.

This paper extends earlier results by Ward, Kennedy and Williamson [1,2] for the design of broadband arrays with frequency-invariant (FI) beam patterns to the case where it is desired to place an...

FIR Approximation of Fractional Sample Delay Systems (1996)

Kootsookos, Peter J., Williamson, Robert C.

This paper examines the approximation of fractional delays by FIR systems using various techniques which have previously been reported in the literature. In particular, the equivalence of the...

Frequency Invariant Broadband Beamforming with Exact Null Placement (1996)

Kootsookos, Peter J., Ward, Darren B., Williamson, Robert C.

This paper extends earlier results by Ward, Kennedy and Williamson [1,2] for the design of broadband arrays with frequency-invariant (FI) beam patterns to the case where it is desired to place an...

FIR Approximation of Fractional Sample Delay Systems (1996)

Kootsookos, Peter J., Williamson, Robert C.

This paper examines the approximation of fractional delays by FIR systems using various techniques which have previously been reported in the literature. In particular, the equivalence of the...

Learning Nonlinearly Parametrized Decision Regions (1996)

Kim L. Blackmore, Robert C. Williamson

In this paper we present a deterministic analysis of an online scheme for learning very general classes of nonlinearly parametrized decision regions. The only input required is a sequence ((xk; yk))...

Learning Nonlinearly Parametrized Decision Regions (1996)

Kim L. Blackmore, Robert C. Williamson

In this paper we present a deterministic analysis of an online scheme for learning very general classes of nonlinearly parametrized decision regions. The only input required is a sequence ((xk; yk))...

Structural Risk Minimization over Data-Dependent Hierarchies (1996)

John Shawe-taylor, Royal Holloway, Peter L. Bartlett, Robert C. Williamson, Martin Anthony

The paper introduces some generalizations of Vapnik's method of structural risk minimisation (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to...

Efficient Agnostic Learning of Neural Networks with Bounded Fan-in (1996)

Wee Sun Lee, Peter L. Bartlett, Robert C. Williamson

We show that the class of two layer neural networks with bounded fan-in is efficiently learnable in a realistic extension to the Probably Approximately Correct (PAC) learning model. In this model, a...

A Framework for Structural Risk Minimisation (1996)

John Shawe-taylor, Royal Holloway, Peter L. Bartlett, Systems Engineering Dept, Robert C. Williamson, Engineering Dept, ...

The paper introduces a framework for studying structural risk minimisation. The model views structural risk minimisation in a PAC context. It then considers the more general case when the hierarchy...

Structural Risk Minimization over Data-Dependent Hierarchies (1996)

John Shawe-taylor, Peter L. Bartlett, Robert C. Williamson, Martin Anthony

The paper introduces some generalizations of Vapnik's method of structural risk minimisation (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to...

Learning Nonlinearly Parametrized Decision Regions (1996)

Kim Blackmore Robert, Robert C. Williamson

In this paper we present a deterministic analysis of an online scheme for learning very general classes of nonlinearly parametrized decision regions. The only input required is a sequence ((xk ; yk...

Fat-Shattering and the Learnability of Real-Valued Functions (1996)

Philip M. Long, Robert C. Williamson, Peter L. Bartlett, Peter L. Bartlett

We consider the problem of learning real-valued functions from random examples when the function values are corrupted with noise. With mild conditions on independent observation noise, we provide...

Efficient Agnostic Learning of Neural Networks with Bounded Fan-in (1996)

Wee Sun, Peter L. Bartlett, Robert C. Williamson

We show that the class of two layer neural networks with bounded fan-in is efficiently learnable in a realistic extension to the Probably Approximately Correct (PAC) learning model. In this model, a...

A Framework for Structural Risk Minimisation (1996)

John Shawe-taylor, Royal Holloway, Peter L. Bartlett, Robert C. Williamson, Martin Anthony

The paper introduces a framework for studying structural risk minimisation. The model views structural risk minimisation in a PAC context. It then considers the more general case when the hierarchy...

Fat-Shattering and the Learnability of Real-Valued Functions (1996)

Peter L. Bartlett, Philip M. Long, Robert C. Williamson

We consider the problem of learning real-valued functions from random examples when the function values are corrupted with noise. With mild conditions on independent observation noise, we provide...

Decision Making and Adjustment among Soviet-Jewish EmigrEs in a Middle-sized Urban Area (1996)

WILLIAMSON, ROBERT C.

Since the 1970s Soviet-Jewish refugees have increasingly chosen the US over Israel. Most of the earlier migrants settled in large cities, where the émigrés and the Jewish community both felt...

Lower Bounds on the VC-Dimension of Smoothly Parametrized Function Classes (1995)

Wee Sun Lee, Peter L. Bartlett, Robert C. Williamson

We examine the relationship between the VCdimension and the number of parameters of a smoothly parametrized function class. We show that the VC-dimension of such a function class is at least k if...

Lower Bounds on the VC-Dimension of Smoothly Parametrized Function Classes (1995)

Wee Sun Lee, Peter L. Bartlett, Robert C. Williamson

We examine the relationship between the VC-dimension and the number of parameters of a thresholded smoothly parametrized function class. We show that the VC-dimension of such a function class is at...

Fat-Shattering and the Learnability of Real-Valued Functions (1995)

Philip M. Long, Robert C. Williamson, Peter L. Bartlett, Peter L. Bartlett

We consider the problem of learning real-valued functions from random examples when the function values are corrupted with noise. With mild conditions on independent observation noise, we provide...

On Efficient Agnostic Learning of Linear Combinations of Basis Functions (1995)

Wee Sun Lee, Peter L. Bartlett, Robert C. Williamson

We consider efficient agnostic learning of linear combinations of basis functions when the sum of absolute values of the weights of the linear combinations is bounded. With the quadratic loss...

Online Learning via Congregational Gradient Descent (1995)

Kim L. Blackmore, Robert C. Williamson, William A. Sethares

We propose and analyse a populational version of stepwise gradient descent suitable for a wide range of learning problems. The algorithm is motivated by genetic algorithms which update a population...

Threshold Effects in Multiharmonic Maximum Likelihood Frequency Estimation (1994)

Williamson, Robert C., James, Ben, Anderson, Brian D. O., Kootsookos, Peter J.

This paper derives a general expression for the mean square error in estimating the fundamental frequency of a multiharmonic signal from a finite sequence of noisy measurements. The distinguishing...

Threshold Effects in Multiharmonic Maximum Likelihood Frequency Estimation (1994)

Williamson, Robert C., James, Ben, Anderson, Brian D. O., Kootsookos, Peter J.

This paper derives a general expression for the mean square error in estimating the fundamental frequency of a multiharmonic signal from a finite sequence of noisy measurements. The distinguishing...

Threshold Effects in Multiharmonic Maximum Likelihood Frequency Estimation (1994)

Williamson, Robert C., James, Ben, Anderson, Brian D. O., Kootsookos, Peter J.

This paper derives a general expression for the mean square error in estimating the fundamental frequency of a multiharmonic signal from a finite sequence of noisy measurements. The distinguishing...

Threshold Effects in Multiharmonic Maximum Likelihood Frequency Estimation (1994)

Williamson, Robert C., James, Ben, Anderson, Brian D. O., Kootsookos, Peter J.

This paper derives a general expression for the mean square error in estimating the fundamental frequency of a multiharmonic signal from a finite sequence of noisy measurements. The distinguishing...

The Relationship Between Instantaneous Frequency and Time-Frequency Representations (1993)

Lovell, Brian C., Williamson, Robert C., Boashash, Boulem

This paper describes a procedure for the time-frequency analysis of signals based on Time-Frequency Distribution (TFD) and Instantaneous Frequency (IF) estimation. First we use a suitable TFD to...

The Relationship Between Instantaneous Frequency and Time-Frequency Representations (1993)

Lovell, Brian C., Williamson, Robert C., Boashash, Boulem

This paper describes a procedure for the time-frequency analysis of signals based on Time-Frequency Distribution (TFD) and Instantaneous Frequency (IF) estimation. First we use a suitable TFD to...

The Relationship Between Instantaneous Frequency and Time-Frequency Representations (1993)

Lovell, Brian C., Williamson, Robert C., Boashash, Boulem

This paper describes a procedure for the time-frequency analysis of signals based on Time-Frequency Distribution (TFD) and Instantaneous Frequency (IF) estimation. First we use a suitable TFD to...

The Relationship Between Instantaneous Frequency and Time-Frequency Representations (1993)

Lovell, Brian C., Williamson, Robert C., Boashash, Boulem

This paper describes a procedure for the time-frequency analysis of signals based on Time-Frequency Distribution (TFD) and Instantaneous Frequency (IF) estimation. First we use a suitable TFD to...

The VC-Dimension and Pseudodimension of Two-Layer Neural Networks with Discrete Inputs (1993)

Peter Bartlett Robert, Robert C. Williamson

We give upper bounds on the Vapnik-Chervonenkis dimension and pseudodimension of two-layer neural networks that use the standard sigmoid function or radial basis function and have inputs from...

Local Minima and Attractors at Infinity for Gradient Descent Learning Algorithms (1993)

Kim Blackmore Robert, Robert C. Williamson

In the paper "Learning Nonlinearly Parametrized Decision Regions ", an online scheme for learning a very general class of decision regions is given, together with conditions on both the...

Local Minima and Attractors at Infinity for Gradient Descent Learning Algorithms (1993)

Kim Blackmore, Robert C. Williamson

In the paper "Learning Nonlinearly Parametrized Decision Regions ", an online scheme for learning a very general class of decision regions is given, together with conditions on both the...

The VC-Dimension and Pseudodimension of Two-Layer Neural Networks with Discrete Inputs (1993)

Peter Bartlett, Robert C. Williamson

We give upper bounds on the Vapnik-Chervonenkis dimension and pseudodimension of two-layer neural networks that use the standard sigmoid function or radial basis function and have inputs from...

Efficient Frequency Estimation and Time-Frequency Representations (1990)

Lovell, Brian C., Kootsookos, Peter J., Williamson, Robert C.

A computationally efficient estimator of the frequency of a single complex sinusoind in complex white Gaussian noise was recently proposed by Kay. Combining Kay's analysis with our own knowledge, we...

Efficient Frequency Estimation and Time-Frequency Representations (1990)

Lovell, Brian C., Kootsookos, Peter J., Williamson, Robert C.

A computationally efficient estimator of the frequency of a single complex sinusoind in complex white Gaussian noise was recently proposed by Kay. Combining Kay's analysis with our own knowledge, we...

Efficient Frequency Estimation and Time-Frequency Representations (1990)

Lovell, Brian C., Kootsookos, Peter J., Williamson, Robert C.

A computationally efficient estimator of the frequency of a single complex sinusoind in complex white Gaussian noise was recently proposed by Kay. Combining Kay's analysis with our own knowledge, we...

Probabilistic arithmetic / (1989)

Williamson, Robert C. (Robert Charles)

Thesis (Ph. D.)--University of Queensland, 1990.

Efficient Frequency Estimation and Time-Frequency Representations

Lovell, Brian C., Kootsookos, Peter J., Williamson, Robert C., Boashash, Boualem, Boles, Peter

A computationally efficient estimator of the frequency of a single complex sinusoind in complex white Gaussian noise was recently proposed by Kay. Combining Kay's analysis with our own knowledge, we...

Threshold Effects in Multiharmonic Maximum Likelihood Frequency Estimation

Williamson, Robert C., James, Ben, Anderson, Brian D. O., Kootsookos, Peter J.

This paper derives a general expression for the mean square error in estimating the fundamental frequency of a multiharmonic signal from a finite sequence of noisy measurements. The distinguishing...

The Relationship Between Instantaneous Frequency and Time-Frequency Representations

Lovell, Brian C., Williamson, Robert C., Boashash, Boulem

This paper describes a procedure for the time-frequency analysis of signals based on Time-Frequency Distribution (TFD) and Instantaneous Frequency (IF) estimation. First we use a suitable TFD to...

FIR Approximation of Fractional Sample Delay Systems

Kootsookos, Peter J., Williamson, Robert C.

This paper examines the approximation of fractional delays by FIR systems using various techniques which have previously been reported in the literature. In particular, the equivalence of the...

Frequency Invariant Broadband Beamforming with Exact Null Placement

Kootsookos, Peter J., Ward, Darren B., Williamson, Robert C., Giannakis, G. B., Swami, A.

This paper extends earlier results by Ward, Kennedy and Williamson [1,2] for the design of broadband arrays with frequency-invariant (FI) beam patterns to the case where it is desired to place an...

Imposing Pattern Nulls on Broadband Array Responses

Kootsookos, Peter J., Ward, Darren B., Williamson, Robert C.

This paper considers the problem of altering a quiescent design for an array of omni-directional sensors so that the altered design rejects a farfield broadband signal from a given direction. This...

A Generalized Representer Theorem

Bernhard Schölkopf, Ralf Herbrich, Alex J. Smola, Robert C. Williamson

Wahba's classical representer theorem states that the solutions of certain optimization problems involving an empirical risk term and a quadratic regularizer can be written as expansions in term...

The Importance of Convexity in Learning with Squared Loss

Wee Sun, Peter L. Bartlett, Robert C. Williamson

We show that if the closure of a function class F under the metric induced by some probability distribution is not convex, then the sample complexity for agnostically learning F with squared loss...