Ivan Markovsky, Properties Π Π
(Lecture 3) Least squares and the singular value decomposition 1 / 52Outline
• Extensions of the least squares problem University of Southampton • Recursive (2009)
Least squares and the singular value
Ivan Markovsky, Sabine Van Huffel, Bart De Moor
min ∥ ∆A,∆B,X [ ∆A ∆B] ∥ ∥2 F s.t. (A − ∆A)X = B − ∆B and ∆A ∆B has the same structure as A B Approaches: CTLS constraint TLS (Abatzoglou et al., 1991)
• Exact identification: the most powerful unfalsified model
Outline • Exact identification problems The most powerful unfalsified model (2009)
Identifiability Conditions, Ivan Markovsky, R R Rℓ, R R Rℓ, R R Rℓ
• from data to kernel representation • impulse response identification • N4SID-type algorithms • MOESP-type algorithms (Lecture 5) The most powerful unfalsified model 1 / 35 (Lecture 5) The...
(Lecture 5) The most powerful unfalsified model 1 / 35Outline • Exact identification problems (2009)
Ivan Markovsky, Identifiability Conditions, R R Rℓ, R R Rℓ, R R Rℓ, R R Rℓ
• from data to kernel representation • impulse response identification • N4SID-type algorithms • MOESP-type algorithms (Lecture 5) The most powerful unfalsified model 2 / 35Exact...
Identification of Electrically Stimulated Muscle after Stroke (2009)
Le, Fengmin, Markovsky, Ivan, Freeman, Christopher, Rogers, Eric
The design of controllers to enable the application of Functional Electrical Stimulation as part of a rehabilitation programme for stroke patients requires an accurate model of electrically...
Identification of Electrically Stimulated Muscle after Stroke (2009)
Le, Fengmin, Markovsky, Ivan, Freeman, Christopher, Rogers, Eric
The design of controllers to enable the application of Functional Electrical Stimulation as part of a rehabilitation programme for stroke patients requires an accurate model of electrically...
An algorithm for closed-loop data-driven simulation (2009)
Closed-loop data-driven simulation refers to the problem of constructing trajectories of a closed-loop system directly from data of the plant and a representation of the controller. Conditions under...
Applications of structured low-rank approximation (2009)
A number of problems in system theory, signal processing, and computer algebra fit into a generic structured low-rank approximation problem. Several problems of this type are reviewed in this paper...
Total Least Squares Methods (2009)
Markovsky, Ivan, Sima, Diana M., Van Huffel, Sabine
Recent advances in total least squares approaches for solving various errors-in-variables modeling problems are reviewed, with emphasis on the following generalizations: 1. the use of weighted norms...
Approximate low-rank factorization with structured factors (2009)
Markovsky, Ivan, Niranjan, Mahesan
An approximate rank revealing factorization problem with structure constraints on the normalized factors is considered. Examples of structure, motivated by an application in microarray data analysis,...
System Identification of Muscle Dynamics for ILC-based Stroke Rehabilitation (2009)
Le, Fengmin, Markovsky, Ivan, Freeman, Christopher, Rogers, Eric
In this paper results are presented for the identification of electrically stimulated muscle dynamics in stroke patients. This research forms a critical component in the model-based control of...
Matrix centering and low-rank approximation (2009)
Low-rank approximation, better known in the machine learning literature as principal component analysis, is a general data modeling tool. Data centering, on the other hand, is a common preprocessing...
Estimation of clock synchronization errors (2009)
Przedwojski, Marek, Markovsky, Ivan, Rogers, Eric
A discrete-time linear time-invariant system that is decomposed into subsystems is considered. The subsystems' states updates are synchronized by a common clock, however, the clock signal reaches the...
Ivan Markovsky, N. Trefethen, Numerical Linear Algebra, G. Golub, Van Loan, Matrix Computations
2. Numerical linear algebra: matrix factorizations
Least squares contour alignment (2009)
Markovsky, Ivan, Mahmoodi, Sasan
The contour alignment problem, considered in this paper, is to compute the minimal distance in a least squares sense, between two explicitly represented contours, specified by corresponding points,...
Ivan Markovsky, J. C. Willems, P. Rapisarda, B. De Moor, S. Van Huffel
What does it mean “is a trajectory of”? let σ be the shift operator σx(t) = x(t + 1) and let Σ be defined by a state space representation Σ: σx = Ax+Bu, y = Cx+Du (I/S/O) (ud,yd) is a...
K.U.Leuven, ESAT-SISTA Joint work with (2008)
Ivan Markovsky, Jan C. Willems, Sabine Van Huffel, Bart De Moor, Rik Pintelon
Application of structured total least squares for system identification
Ivan Markovsky, G. Strang, Linear Algebra, N. Trefethen, Numerical Linear Algebra, G. Golub, ...
1. Review of linear algebra
State estimation with noisy input and output in this talk, we pose and answer the question: how should we modify the Kalman filter when both the input and the output of the system are noisy? we...
Data-driven simulation and control (2008)
Markovsky, Ivan, Rapisarda, Paolo
Classical linear time-invariant system simulation methods are based on a transfer function, impulse response, or input/state/output representation. We present a method for computing the response of a...
Least squares contour alignment (2008)
Markovsky, Ivan, Mahmoodi, Sasan
The contour alignment problem, considered in this paper, is to compute the minimal distance in a least squares sense, between two explicitly represented contours, specified by corresponding points,...
Unfalsified control of linear time-invariant systems (2008)
The unfalsified control concept is a data-driven assumptions-free control strategy. Its main tool is a controller falsification procedure, which tests the performance of a candidate controller...
Identification of the Dynamics of Human Arms after Stroke (2008)
Le, Fengmin, Markovsky, Ivan, Freeman, Christopher, Rogers, Eric
The design of controllers to enable the application of Functional Electrical Stimulation as part of a rehabilitation programme for stroke patients requires an accurate model of electrically...
Results on the PASCAL challenge "Simple causal effects in time series" (2008)
A solution to the PASCAL challenge ``Simple causal effects in time series'' (www.causality.inf.ethz.ch) is presented. The data is modeled as a sum of a constant-plus-sin term and a term that is a...
Least squares contour alignment (2008)
Markovsky, Ivan, Mahmoodi, Sasan
The contour alignment problem, considered in this paper, is to compute the minimal distance in a least squares sense, between two explicitly represented contours, specified by corresponding points,...
Ivan Markovsky, J. C. Willems, P. Rapisarda, B. De Moor, S. Van Huffel
An exact identification problem Problem P1 (Exact identification) Given two vector time series ud = � ud(1),...,ud(T) � ∈ (R m) T yd = � yd(1),...,yd(T) � ∈ (R p) T “inputs”...
Markovsky, Ivan, Amann, Anton, Van Huffel, Sabine
Band-pass, Kalman, and adaptive filters are used for removal of resuscitation artifacts from human ECG signals. A database of separately recorded human ECG and animal resuscitation artifact signals...
Rapisarda, Paolo, Markovsky, Ivan
We study the linear quadratic control problem from a representation-free point of view, and we show that this formulation brings forth two self-contained and original proofs of the optimality of...
Approximate low-rank factorization with structured factors (2008)
Markovsky, Ivan, Niranjan, Mahesan
The notion of a matrix rank is of fundamental importance in data analysis, however, rank estimation is known to be a notoriously difficult problem in numerical computing. A rank estimation procedure...
Palindromic polynomials, time-reversible systems, and conserved quantities (2008)
The roots of palindromic and antipalindromic polynomials appear in pairs (s,1/s). A polynomial with such roots is antipalindromic if and only if in addition, it has a root at 1 of an odd...
Ivan Markovsky, Ilse Pardon (accountant, Bart Motmans (coordinator
• Scientific research network “Advanced numerical methods for mathematical modelling”
Results on the PASCAL challenge "Simple causal effects in time series" (2008)
A solution to the PASCAL challenge ``Simple causal effects in time series'' (www.causality.inf.ethz.ch) is presented. The data is modeled as a sum of a constant-plus-sin term and a term that is a...
Computation of LTI system responses directly from input/output data 1 (2007)
Ivan Markovsky, Jan C. Willems, Paolo Rapisarda, Bart L. M, De Moor
Submitted for publication to CDC 2004
Published in the proc. of Conference on Decision and Control (CDC 2002) (2007)
Ivan Markovsky, Jan C. Willems, B. De Moor
pages 2576--2581
Multi-Model System Parameter Estimation (2007)
Ivan Markovsky, Sabine Van Huffel, Bart De Moor
We pose a multi-model system parameter estimation problem. A multi-model system is a linearly parameterized system H(z, p) = #np i=1 p i H i (z). The parameter estimation problem is: given the set of...
of the Belgian Programme on Interuniversity Poles of Attraction (IUAP (2007)
Er Kukush, Ivan Markovsky, Sabine Van Huel
This report is available by anonymous ftp from ftp.esat.kuleuven.ac.be in the directory pub/sista/markovsky/reports/02-49.ps.gz 2
Submitted for publication to Automatica 1 (2007)
This report is available by anonymous ftp from ftp.esat.kuleuven.ac.be in the directory pub/sista/markovsky/reports/03-183.ps.gz 2
Estimation in a linear multivariate measurement error model with a change point in the data (2007)
Kukush, Alexander, Markovsky, Ivan, Van Huffel, Sabine
A linear multivariate measurement error model AX=B is considered. The errors in [A B] are row-wise finite dependent, and within each row, the errors may be correlated. Some of the columns may be...
Estimation in a linear multivariate measurement error model with a change point in the data (2007)
Kukush, Alexander, Markovsky, Ivan, Van Huffel, Sabine
A linear multivariate measurement error model AX=B is considered. The errors in [A B] are row-wise finite dependent, and within each row, the errors may be correlated. Some of the columns may be...
Palindromic polynomials, time-reversible systems, and conserved quantities (2007)
The roots of palindromic and antipalindromic polynomials can be grouped in pairs $(\lambda,1/\lambda)$. A polynomial with such root pattern is palindromic/antipalindromic if, in addition, it has a...
Palindromic polynomials, time-reversible systems, and conserved quantities (2007)
The roots of palindromic and antipalindromic polynomials can be grouped in pairs (\lambda,1/\lambda). A polynomial with such root pattern is palindromic/antipalindromic if, in addition, it has a root...
On the optimality of state feedback in linear quadratic control (2007)
Rapisarda, Paolo, Markovsky, Ivan
We study the linear quadratic control problem from a representation-free point of view, and we show that this formulation brings forth several results of independent interest. Among these results are...
On the optimality of state feedback in linear quadratic control (2007)
Rapisarda, Paolo, Markovsky, Ivan
We study the linear quadratic control problem from a representation-free point of view, and we show that this formulation brings forth several results of independent interest. Among these results are...
On the linear quadratic data-driven control (2007)
Markovsky, Ivan, Rapisarda, Paolo
The classical approach for solving control problems is model based: first a model representation is derived from given data of the plant and then a control law is synthesized using the model and the...
On the linear quadratic data-driven control (2007)
Markovsky, Ivan, Rapisarda, Paolo
The classical approach for solving control problems is model based: first a model representation is derived from given data of the plant and then a control law is synthesized using the model and the...
Left vs right representations for solving weighted low-rank approximation problems (2007)
Markovsky, Ivan, Van Huffel, Sanine
The weighted low-rank approximation problem in general has no analytical solution in terms of the singular value decomposition and is solved numerically using optimization methods. Four...
Structured low-rank approximation and its applications (2007)
Fitting data by a bounded complexity linear model is equivalent to low-rank approximation of a matrix constructed from the data. The data matrix being Hankel structured is equivalent to the existence...
Left vs right representations for solving weighted low-rank approximation problems (2007)
Markovsky, Ivan, Van Huffel, Sanine
The weighted low-rank approximation problem in general has no analytical solution in terms of the singular value decomposition and is solved numerically using optimization methods. Four...
Structured low-rank approximation and its applications (2007)
Fitting data by a bounded complexity linear model is equivalent to low-rank approximation of a matrix constructed from the data. The data matrix being Hankel structured is equivalent to the existence...
Left vs right representations for solving weighted low-rank approximation problems (2007)
Markovsky, Ivan, Van Huffel, Sanine
The weighted low-rank approximation problem in general has no analytical solution in terms of the singular value decomposition and is solved numerically using optimization methods. Four...
On the conic section fitting problem (2007)
Kukush, Alexander, Markovsky, Ivan
Adjusted least squares (ALS) estimators for the conic section problem are considered. Consistency of the translation invariant version of ALS estimator is proved. The similarity invariance of the ALS...
Structured low-rank approximation and its applications (2007)
Fitting data by a bounded complexity linear model is equivalent to low-rank approximation of a matrix constructed from the data. The data matrix being Hankel structured is equivalent to the existence...
Data-driven simulation and control (2007)
Markovsky, Ivan, Rapisarda, Paolo
Classical linear time-invariant system simulation methods are based on a transfer function, impulse response, or input/state/output representation. We present a method for computing the response of a...
Data-driven simulation and control (2007)
Markovsky, Ivan, Rapisarda, Paolo
Classical linear time-invariant system simulation methods are based on a transfer function, impulse response, or input/state/output representation. We present a method for computing the response of a...
Schuermans, Mieke, Markovsky, Ivan, Van Huffel, Sabine
The Maximum Likelihood PCA (MLPCA) method has been devised in chemometrics as a generalization of the well-known PCA method in order to derive consistent estimators in the presence of errors with...
Overview of total least squares methods (2007)
Markovsky, Ivan, Van Hufel, Sabine
We review the development and extensions of the classical total least squares method and describe algorithms for its generalization to weighted and structured approximation problems. In the generic...
Estimation in a linear multivariate measurement error model with a change point in the data (2007)
Kukush, Alexander, Markovsky, Ivan, Van Huffel, Sabine
A linear multivariate measurement error model AX=B is considered. The errors in [A B] are row-wise finite dependent, and within each row, the errors may be correlated. Some of the columns may be...
Schuermans, Mieke, Markovsky, Ivan, Van Huffel, Sabine
The Maximum Likelihood PCA (MLPCA) method has been devised in chemometrics as a generalization of the well-known PCA method in order to derive consistent estimators in the presence of errors with...
Schuermans, Mieke, Markovsky, Ivan, Van Huffel, Sabine
The Maximum Likelihood PCA (MLPCA) method has been devised in chemometrics as a generalization of the well-known PCA method in order to derive consistent estimators in the presence of errors with...
Overview of total least squares methods (2007)
Markovsky, Ivan, Van Huffel, Sabine
We review the development and extensions of the classical total least squares method and describe algorithms for its generalization to weighted and structured approximation problems. In the generic...
Overview of total least squares methods (2007)
Markovsky, Ivan, Van Hufel, Sabine
We review the development and extensions of the classical total least squares method and describe algorithms for its generalization to weighted and structured approximation problems. In the generic...
Structured low-rank approximation and its applications (2007)
Fitting data by a bounded complexity linear model is equivalent to low-rank approximation of a matrix con-structed from the data. The data matrix being Hankel structured is equivalent to the...
On weighted structured total least squares (2006)
Markovsky, Ivan, Van Huffel, Sabine
In this contribution we extend the result of (Markovsky et. al, SIAM J. of Matrix Anal. and Appl., 2005) to the case of weighted cost function. It is shown that the computational complexity of the...
On weighted structured total least squares (2006)
Markovsky, Ivan, Van Huffel, Sabine
In this contribution we extend the result of (Markovsky et. al, SIAM J. of Matrix Anal. and Appl., 2005) to the case of weighted cost function. It is shown that the computational complexity of the...
On weighted structured total least squares (2006)
Markovsky, Ivan, Van Huffel, Sabine
In this contribution we extend the result of (Markovsky et. al, SIAM J. of Matrix Anal. and Appl., 2005) to the case of weighted cost function. It is shown that the computational complexity of the...
A note on persistency of excitation (2005)
Willems, Jan C., Rapisarda, Paolo, Markovsky, Ivan
We prove that if a component of the response signal of a controllable linear time-invariant system is persistently exciting of sufficiently high order, then the windows of the signal span the full...
Algorithms for deterministic balanced subspace identification (2005)
Markovsky, Ivan, Willems, Jan C., Rapisarda, Paolo
New algorithms for identification of a balanced state space representation are proposed. They are based on a procedure for the estimation of impulse response and sequential zero input responses...
A note on persistency of excitation (2005)
Willems, Jan C., Rapisarda, Paolo, Markovsky, Ivan
We prove that if a component of the response signal of a controllable linear time-invariant system is persistently exciting of sufficiently high order, then the windows of the signal span the full...
Algorithms for deterministic balanced subspace identification (2005)
Markovsky, Ivan, Willems, Jan C., Rapisarda, Paolo
New algorithms for identification of a balanced state space representation are proposed. They are based on a procedure for the estimation of impulse response and sequential zero input responses...
A note on persistency of excitation (2005)
Willems, Jan C., Rapisarda, Paolo, Markovsky, Ivan
We prove that if a component of the response signal of a controllable linear time-invariant system is persistently exciting of sufficiently high order, then the windows of the signal span the full...
Algorithms for deterministic balanced subspace identification (2005)
Markovsky, Ivan, Willems, Jan C., Rapisarda, Paolo
New algorithms for identification of a balanced state space representation are proposed. They are based on a procedure for the estimation of impulse response and sequential zero input responses...
Structured matrices given a injective mapping S: R np → R m×(n+d) , we say that the matrix C ∈ R m×n+d is S-structured if C ∈ image(S) let C be S-structured, then the vector p ∈ R np, such...
∆A,∆B,X � ∆A ∆B � � �2 s.t. (A − ∆A)X = B − ∆B and F (2004)
Sabine Van Huffel, Ivan Markovsky
Software package for
A Note On Persistency Of Excitation (2004)
Jan C. Willems, Paolo Rapisarda, Ivan Markovsky, Bart De Moor
This report is available by anonymous ftp from ftp.esat.kuleuven.ac.be in the directory pub/sista/markovsky/reports/04-101.ps.gz K.U.Leuven, Dept. of Electrical Engineering (ESAT), Research group SCD...
Application of Structured Total Least Squares for System Identification and Model Reduction (2004)
Ivan Markovsky, Sabine Van Huffel, Jan C. Willems, Sabine Van Hu#el, Bart De Moor, ...
This report is available by anonymous ftp from ftp.esat.kuleuven.ac.be in the directory pub/sista/markovsky/reports/04-51.ps.gz K.U.Leuven, Dept. of Electrical Engineering (ESAT), Research group SCD...
Application of Structured Total Least Squares for System Identification (2004)
Ivan Markovsky, Sabine Van Huffel, Rik Pintelon, Sabine Van Hu#el
The following identification problem is considered: minimize the # 2 norm of the di#erence between a given time series and an estimated one under the constraint that the estimated time series is a...
Software for structured total least squares estimation: User's guide (2003)
Abstract. ± We consider the problem of solving approximately an overdetermined system of linear equations AX B, where the data matrix AB is structured. Block-Hankel structured approximation problems...
High-performance numerical algorithms and software for structured total least squares (2003)
Ivan Markovsky, Sabine Van Huffel
www.elsevier.com/locate/cam
Linear dynamic filtering with noisy input and output (2002)
We establish the equivalence between the optimal least-squares state estimator for a linear time-invariant dynamic system with noise corrupted input and output, and an appropriately modified Kalman...
Alexander Kukush, Sabine Van Huffel, Er Kukush, Ivan Markovsky, Sabine Van Hu#el
This report is available by anonymous ftp from ftp.esat.kuleuven.ac.be in the directory pub/sista/markovsky/reports/02-192.ps.gz K.U.Leuven, Dept. of Electrical Engineering (ESAT), Research group SCD...
On the Computation of the Multivariate Structured Total Least Squares Estimator (2002)
Sabine Van Huffel, Alexander Kukush, Sabine Van Hu#el, Er Kukush, Ivan Markovsky, ...
A multivariate structured total least squares problem is considered, in which the extended data matrix is partitioned into blocks and each of the blocks is Toeplitz/Hankel structured, unstructured,...
The element-wise weighted total least-squares problem
Markovsky, Ivan, Luisa Rastello, Maria, Premoli, Amedeo, Kukush, Alexander, Van Huffel, Sabine
On the conic section fitting problem
Shklyar, Sergiy, Kukush, Alexander, Markovsky, Ivan, Van Huffel, Sabine
Adjusted least squares (ALS) estimators for the conic section problem are considered. Consistency of the translation invariant version of ALS estimator is proved. The similarity invariance of the ALS...