Bayesian Methods
Course topics
Part 1. Introduction to the Bayesian theory
- Principles, uncertainty, prior/posterior knowledge, hyperparameters, likelihood, Bayes theorem, difference to frequentist statistics, sequential (online) modeling. Types of estimates.
- Prior and posterior distribution, the existence of analytical solutions, conjugacy, exponential family, real-world examples
- Linear regression and autoregression, types of reality-reflecting models (black box, grey box), construction of linear models based on physical principles
- Basics of Bayesian estimation with conjugate prior distributions (Practical classes)
- Sequential (online) linear regression and autoregression with real data. Exponential forgetting in the estimation of models with time-varying parameters (Practical classes)
Part 2. Generalized linear models
- Logistic regression and online logistic regression, MAP estimation. Sketch of other GLMs. Practical examples
- State-space models I
- Evolution of model parameters (i.e., states). Linear state-space models, Kalman filtering. Real-world examples (Apollo program, free fall equation, target tracking)
- Sequential logistic regression (Practical classes)
- Kalman filtering (Practical classes)
Part 3. State-space models II/Advanced methods
- Nonlinear models and introduction to particle filtering. Principles of particle filters, filter degeneracy, resampling
- Discussion of principles of selected advanced methods like Dynamic Model Averaging, Approximate Bayesian Computation and/or others based on students’ interests
- Particle filtering (Practical classes)
- Concluding discussion
Prerequisites
- Linear algebra
- Mathematical analysis
- Statistics and Probability Theory
- Programming: basic programming skills on Python