Course Homepage for Advanced Computational Statistics (PhD Course 2023; 7.5 HEC)

Summary

Statistics depends heavily on computational methods. Optimisation methods are used in statistics for example for maximum likelihood estimates, optimal experimental designs, risk minimization in decision theoretic models. In these cases, solutions of optimisation problems usually do not have a closed form but need to be computed numerically with an algorithm. Another big demand on computational methods is when statistical distributions are simulated and integrated and statistics of these distributions have to be determined in an efficient way.

This course focuses on computational methods for optimisation, simulation and integration needed in statistics. The optimisation part discusses gradient based, stochastic gradient based, and gradient free methods. Further, constrained optimisation will be a course topic. We will discuss techniques to simulate efficiently for solving statistical problems.

We will use implementation with the programming language R. Examples from machine learning and optimal design will illustrate the methods.

Most welcome to the course!
Frank Miller, Department of Computer and Information Science, Linköping University
frank.miller at liu.se

Prerequisites

Accepted to a doctoral program in Sweden in Statistics or a related field (e.g. Mathematical Statistics, Engineering Science, Quantitative Finance, Computer Science). Knowledge about Statistical Inference (e.g. from the Master's level) and familiarity with a programming language (e.g. with R) is required.

Note that the content of the optimisation part of the course will have considerable overlap with the Ph.D. courses Optimisation algorithms in Statistics I/II which I gave in 2020/2021, i.e. the current course is not intended for the participants of the former courses.

Content

The course contains fundamental principles of computational statistics. Focus is on:

  • Principles of gradient based and gradient free optimisation including stochastic optimisation and constrained optimisation
  • Introduction to convergence analysis for stochastic optimisation algorithms
  • Statistical problem-solving using optimisation, including maximum likelihood, regularized least squares, and optimal experimental designs
  • Principles of numerical integration
  • Principles of statistical simulation
  • The bootstrap method
  • Statistical problem-solving using simulation techniques including generation of Monte Carlo estimates, their confidence intervals, and posterior distributions

Intended Learning Outcomes

On completion of the course, the student is expected to be able to:

  • Demonstrate knowledge of principles of computational statistics
  • Explain theoretical and empirical methods to compare different algorithms
  • Design and organize algorithms for optimisation, integration, and simulation of distributions
  • Solve statistical computing problems using advanced algorithms
  • Adapt a given optimisation, integration, or simulation method to a specific problem
  • Assess, compare and contrast properties of alternative optimisation, integration, and simulation methods
  • Critically judge different methods for optimisation, integration, and simulation
  • Ability to choose an adequate method for a given statistical problem

Examination and Grading

The intended learning outcomes will be graded by several individual home assignments. The grades given: Pass or Fail.

Course Literature
  • Givens GH, Hoeting JA (2013). Computational Statistics, 2nd edition. John Wiley & Sons, Inc., Hoboken, New Jersey.
  • Goodfellow I, Bengio Y, Courville A (2016). Deep Learning. MIT Press, http://www.deeplearningbook.org. Focus on Chapter 4, 5, and 8.
  • Further literature including research articles and other learning material will be provided in the course.

Course Structure and Schedule

Lectures and some problem sessions. The teaching is conducted in English. Course participants will spend most of their study time by solving the problem sets for each topic on their own computers without supervision. The course will be held in March, April, and May 2023.

  • Lecture 1: Gradient based optimisation
    March 16, 13:00-17:00 (in Linköping)
  • Lecture 2: Stochastic gradient based optimisation
    March 17, 9:00-12:00 (in Linköping)
  • Lecture 3: Gradient free optimisation
    April 4, 10:00-12:00, 13:00-15:00 (online, Zoom)
  • Lecture 4: Optimisation with constraints
    April 18, 10:00-12:00, 13:00-15:00 (online, Zoom)
  • Lecture 5: EM algorithm and bootstrap
    May 2, 10:00-13:00 (online, Zoom)
  • Lecture 6: Simulation of random variables
    May 16, 13:00-17:00 (probably in Stockholm; alternatively in Linköping)
  • Lecture 7: Importance sampling
    May 17, 9:00-12:00 (probably in Stockholm; alternatively in Linköping)

Registration

To register for the course, please send an email to me (frank.miller at liu.se) until February 20, 2023. You are also welcome for any questions related to the course.