Afternoon Tutorial 1

• Abstract: Mathematical models of dynamical systems are used for analysis, simulation, prediction, optimization, monitoring, fault detection, signal modelling, filtering and detection, training and control. Engineering solutions in telecommunications, biomedical signal processing, industrial process control etc· are increasingly based on mathematical models of systems. For instance, many problems in biomedical and telecommunications signal processing can be phrased as a model-based filtering and/or detection problem. The design of industrial process control systems, especially for multivariable ones, is critically based upon a reliable model of the process at hand.

  It goes without saying that state-space models are good engineering models in many of these applications.

  In this tutorial, we give a survey of so-called subspace identification techniques for multivariable linear time-invariant models (LTI) of the form

  x(k+1) = A x(k) + B u(k) + w(k),
  y(k) = C x(k) + D u(k) + v(k),

  where k is the discrete time index, u(k) and y(k) are the (observed, measured) inputs respectively outputs of the system, x(k) is the (unknown) systemâs state, w(k) and v(k) are unobserved white noise inputs (process noise, resp. measurement noise).

  The identification problem now consists of finding the real model matrices A, B, C and D and the noise covariance matrices from observations of the inputs u(k) and outputs y(k) only, when a sufficient amount of data is available ( k à infinity).

  Since the beginning of the nineties, there has been an increasing interest (witnessed by numerous papers and books) in so-called subspace identification methods, in which geometrical, algebraic and numerical concepts and algorithms are elegantly intertwined, leading to extremely user-friendly software for obtaining LTI models from input-output data.

  It is the purpose of this tutorial to

  • introduce the audience to the basic geometrical, algebraic and numerical ingredients of subspace methods;
  • organise the conceptual understanding of the building blocks of subspace algorithms, so that the user can Îunderstandâ the different variants that have been described in the literature;
  • organise the conceptual understanding of the building blocks of subspace algorithms, so that the user can Îunderstandâ the different variants that have been described in the literature;
  • provide the attending researcher with a state-of-the-art review of the most relevant milestones in the development of subspace methods as well as survey recently obtained results and current research activities in subspace identification;

  Subspace methods derive their user-friendliness and robustness from the fact that

  • There is no need for a user-defined parametrization, the determination of which is a notoriously difficult problem for MIMO systems; In a subspace method, typically, one of the only critical parameters to be determined is the number of states; Said in other words, subspace methods lead to fully (over-)parametrized models (but the overparametrization does not lead to additional difficulties). As an additional feature, there is no need to parametrize the initial state (as is the case with identification methods that are based on input-output models);
  • Subspace methods have led to new conceptual insights in (linear) system identification, such as the insight on how to obtain a (Kalman filter) state estimate directly from inputs and outputs, or on how to obtain reduced models Îdirectlyâ (i.e. without calculating the Îlargerâ model first);
  • In subspace methods, there are typically three basic conceptual steps:
    • First an estimate of the (Kalman filter) state sequence and/or the column space of the observability matrix is obtained; The number of states derives from the rank of certain oblique and orthogonal projections of subspaces generated by the data;
    • Next the model matrices are obtained by solving a least squares problem;
    • The noise covariance matrices are estimated from the least squares residuals;

  There is a large number of possible variations on and within these basic steps, each of which leads to a variant that has been described in the literature.

  • Each of these steps can be robustly and efficiently implemented, resulting in the fact that subspace algorithms are Îniceâ, consisting of well understood basic building blocks such as the QR and the singular value decomposition, hence avoiding problems of ill convergence as in non-linear iterative optimization algorithms;

  This 3-hour tutorial is aimed at providing both a conceptual insight in subspace identification methods as well as a survey of more recent developments. The intended audience consists of

  • PhD students and researchers active in system identification;
  • Regular or occasional users of system identification tools (signal processing and/or control engineers), both from academia or industry, who want to get acquainted with
    • The basic ideas of subspace identification techniques;
    • The user-friendliness and robustness of subspace software;

• Reference:
  Peter Van Overschee, Bart De Moor, Subspace Identification for Linear Systems: Theory, Implementation and Applications, Kluwer Academic Publishers, 1996, p. 254, ISBN 0-7923-9717-7, (http://www.wkap.nl). Matlab floppy disk with subspace identification m-files included.

• Outline:
  • Part I: Subspace identification for deterministic and stochastic systems (1 hour)
    • Basic ingredients of subspace methods
      • Geometry:
        • Row and column spaces of matrices, dimension;
        • Orthogonal and oblique projections;
      • System theory:
        • Block Hankel matrices with inputs and outputs;
        • Realization algorithms;
        • The Kalman filter state directly from inputs and outputs;
      • Numerical issues
        • QR-decomposition;
        • Rank deficiency;
        • Singular Value decomposition;
    • Subspace identification of deterministic systems
      • The state as intersection between past and future;
      • Calculating the model matrices;
    • Subspace identification of stochastic systems
      • Backward and forward innovation models;
      • The Kalman filter state estimate as an orthogonal projection from the future onto the past;
      • Canonical correlations;
      • The issue of positive realness;

  • Part II: Subspace identification for combined systems (1 hour)
    • The Kalman filter state estimate as an oblique projection;
    • Computing the model matrices and noise covariances;
    • Weighted oblique projections and special cases (MOESP, N4SID, CVA, others described in literature,·)
    • Direct model reduction; Ennsâs conjecture;
    • User choices;

  • Part III: Implementations, applications, extensions and open problems (1 hour)
    • Implementations:
      • How to numerically implement robust reliable software for subspace identificaton; Numerical issues; large problems (large number of inputs, outputs, states); Updating issues; Graphical User Interfaces;
      • Survey of subspace identification codes, both the commercially available ones (Matlab, Xmath, Adaptics,·) as the ones available in public domain (SLICOT/NICONET,·);

    • Applications
      • We will illustrate properties and results with applications from process industry, mechatronics and signal processing from datasets publically available at http://www.esat.kuleuven.ac.be/sista/daisy.

    • Extensions
      • Review of extensions to periodic systems, bilinear system identification, system identification for nonlinear system (e.g. Wiener/Hammerstein), frequency domain versions, closed-loop subspace identification, including a priori information (e.g. enforcing stability,·), blind identification, stochastic identification with non-stationary inputs;

    • Recent results and open problems
      • Here we give a short (literature) survey with respect to the current research on subspace identification (including some recently obtained statistical, system theoretic (algebraic and positive degree), genericity (input conditions) and numerical results); We also formulate the most challenging open problems;

• Speaker's Biographies:

  • Bart De Moor (http://www.esat.kuleuven.ac.be/~demoor) is a senior research associate of the Fund for Scientific Research Flanders (FWO-Vlaanderen) and associate professor at the Department of Electrical Engineering of the Katholieke Universiteit Leuven in Belgium. He was born on Tuesday July 12, 1960 and received his doctoral degree in Applied Sciences in 1988 at the Katholieke Universiteit Leuven, Belgium. He was a visiting research associate (1988-1989) at the Departments of Computer Science (Gene Golub) and Electrical Engineering (Thomas Kailath) of Stanford University, California, USA. His research interests include numerical linear algebra (generalized svd, complementarity problems, structured total least squares, tensor algebra), system identification (subspace methods), control theory (robust control, neural nets) and signal processing. He has more than 200 papers in international journals and conference proceedings and received several national and international awards for his work (Leybold-Heraeus Prize (1986), the Leslie Fox Prize (1989), the Guillemin-Cauer Best Paper Award (1990) of the IEEE Transactions on Circuits and Systems, the bi-annual Siemens prize (1994), the tri-annual best paper award of Automatica, a journal of the International Federation of Automatic Control) and he was a Laureate of the Belgian Royal Academy of Sciences (1992). He is a member of several boards of administrators of (inter)national scientific, cultural and commercial organisations, including the Flemish Interuniversity Institute for Biotechnology, the Flemish Center for Postharvest Technology and ISMC NV (Intelligent Systems, Modelling and Control), a spin-off company specialized in modelling and control of multivariable industrial and biotechnological processes and mechatronics analysis and design. He is also a founding member of the philosophical think tank ÎWorldviewsâ. Past appointments include the membership of the board of administrators of the European Control Association and the Belgian Institute for Control and Automation. In 1991-1992 he was the chief of staff of the Belgian federal minister of science Ms. Wivina Demeester-DeMeyer and later on of the Belgian prime minister Wilfried Martens. Since 1994 he is the Advisor on Science and Technology policy of the Flemish minister-president Luc Van den Brande.

  • Peter Van Overschee (http://www.esat.kuleuven.ac.be/~vanovers/) was born in Leuven, Brabant, Belgium on October 28, 1966. He obtained his degree of Electro-Mechanical Engineering in control theory in 1989 at the Katholieke Universiteit Leuven, Belgium. In 1990 he received his Master of Science in Electrical Engineering at Stanford University, California, USA. At the Katholieke Universiteit Leuven, he obtained his doctoral degree with the work "Subspace Identification: Theory - Implementation - Applications" in 1995, which was published as a book by Kluwer in 1996. In 1994, he won the biannual Belgian Siemens Award and in 1996 the tri-annual best paper award of Automatica, a journal of the International Federation of Automatic Control (at the IFAC World Congress, San Francisco, 1996). His current research interests are the theory and application of multivariable system identification. He was also involved in the design, development and implementation of the identification toolbox for Xmath (Integrated Systems Inc., Santa Clara, USA). Currently he is the managing director of the company "Intelligent System Modeling and Control" where he applies advanced modeling and control techniques in industry.

< Return to overview of Tutorials.

Last Update:  February 4, 1999         Ingo Höntsch