Brief summary of the course
The subject is focused on the practical use of basic Bayesian modeling methods in the dynamically evolving machine learning theory. In particular, it studies the construction of appropriate models providing a description of real phenomena, as well as their subsequent use, e.g., for forecasting of future evolution or learning about the hidden variables (true object position from noisy observations, etc.). The emphasis is put on understanding explained principles and methods and their practical adoption. For this purpose, a number of examples and applications will be presented to students, for instance, 2D object tracking, classification from streaming data, etc. The students will try to solve some of them.
Course topics
Part 1. Introduction to the Bayesian theory
- Principles, uncertainty, prior/posterior knowledge, hyperparameters, likelihood, Bayes theorem, difference to frequentist statistics, sequential (online) modeling. Types of estimates.
- Prior and posterior distribution, the existence of analytical solutions, conjugacy, exponential family, real-world examples
- Linear regression and autoregression, types of reality-reflecting models (black box, grey box), construction of linear models based on physical principles
- Basics of Bayesian estimation with conjugate prior distributions (Practical classes)
- Sequential (online) linear regression and autoregression with real data. Exponential forgetting in the estimation of models with time-varying parameters (Practical classes)
Part 2. Generalized linear models
- Logistic regression and online logistic regression, MAP estimation. Sketch of other GLMs. Practical examples
- State-space models I
- Evolution of model parameters (i.e., states). Linear state-space models, Kalman filtering. Real-world examples (Apollo program, free fall equation, target tracking)
- Sequential logistic regression (Practical classes)
- Kalman filtering (Practical classes)
Part 3. State-space models II/Advanced methods
- Nonlinear models and introduction to particle filtering. Principles of particle filters, filter degeneracy, resampling
- Discussion of principles of selected advanced methods like Dynamic Model Averaging, Approximate Bayesian Computation and/or others based on students’ interests
- Particle filtering (Practical classes)
- Concluding discussion
Prerequisites
- Calculus – elementary level
- Linear algebra – elementary level
- Probability and statistics – elementary level