In this chapter, we refresh some basic statistical concepts commonly used in system identification for readers who want to update their statistical background. We illustrate the use of the sample mean, the sample variance, and the interpretation and use of the covariance matrix. We also take the opportunity to introduce a few concepts that are frequently used in system identification like degrees-of-freedom and bias-variance trade-off.
No explicit list of references is provided in this chapter, we refer the reader to standard textbooks on statistics if more information is needed.
The following hands-on exercises guide the reader through the statistical tools:
Sample Mean and Median
In this section, we give a basic introduction to the sample mean and median for the readers who are less familiar with statistics. We emphasize those aspects that will be important to interpret the results from system identification and we also use the opportunity to introduce estimators which is a basic concept in System Identification.
What you will learn:
– Sample mean and sample median,
– Study of the stochastic properties of the sample mean and median,
– A system identification interpretation: Least Squares and Least Absolute Values,
– Discussion of the optimality of the sample mean and median as an estimator for the mean of a distribution.
Download the MATLAB® live script Sample Mean to run the session.
A pdf version of the file can be downloaded here.
Sample Variance
In this section, we provide a basic introduction to the sample variance for readers who are less familiar with statistics. We emphasize those aspects that will be important to interpret system identification results. We also take the opportunity to introduce the concepts degrees-of-freedom and bias-variance trade-off.
What you will learn:
– Definition of the sample variance,
– Study of the properties of the sample variance, using respectively the exact mean and the sample mean,
– Study of the chi-square distribution: introduction of the degrees-of-freedom concept that has an important role in system identification,
– Calculation of confidence intervals on the sample variance,
– Study of the bias-variance trade-off that balances the systematic (bias) and the variability (variance) errors of an estimator.
Download the MATLAB® live script Sample Variance to run the session.
A pdf version of the file can be downloaded here.
Covariance Matrix
In this Hands-On session, we offer a basic introduction to the covariance matrix and its use in system identification for the readers who are less familiar with statistics.
What you will learn:
– Introduction of the covariance matrix,
– Understanding the role of the variance and co-variance,
– Introduction of the correlation matrix,
– Studying the variance of a multivariate function,
– Difference between correlated and independent variables.
Download the MATLAB® live script Covariance Matrix to run the session.
A pdf version of the file can be downloaded here.