Bayesian Methods

Course topics

Part 1. Introduction to the Bayesian theory

  • Principles, uncertainty, prior/posterior knowledge, hyperparameters, likelihood, Bayes theorem, difference to frequentist statistics, sequential (online) modeling. Types of estimates.
  • Prior and posterior distribution, the existence of analytical solutions, conjugacy, exponential family, real-world examples
  • Linear regression and autoregression, types of reality-reflecting models (black box, grey box), construction of linear models based on physical principles
  • Basics of Bayesian estimation with conjugate prior distributions (Practical classes)
  • Sequential (online) linear regression and autoregression with real data. Exponential forgetting in the estimation of models with time-varying parameters (Practical classes)

Part 2. Generalized linear models

  • Logistic regression and online logistic regression, MAP estimation. Sketch of other GLMs. Practical examples
  • State-space models I
  • Evolution of model parameters (i.e., states). Linear state-space models, Kalman filtering. Real-world examples (Apollo program, free fall equation, target tracking)
  • Sequential logistic regression (Practical classes)
  • Kalman filtering (Practical classes)

Part 3. State-space models II/Advanced methods

  • Nonlinear models and introduction to particle filtering. Principles of particle filters, filter degeneracy, resampling
  • Discussion of principles of selected advanced methods like Dynamic Model Averaging, Approximate Bayesian Computation and/or others based on students’ interests
  • Particle filtering (Practical classes)
  • Concluding discussion