About

The course is devoted to the problem of optimal model selection in machine learning and data analysis tasks. During the course, we investigate the Bayesian approach to model selection, its theoretical and practical aspects. We consider properties of model selection methods both specific to some notable model families or general properties suitable for the most model families.

Syllabus

The course investigates the properties of model selection for different types of models: from linear to mixtures of experts built on deep learning models. Various approaches to model selection and hyperparameter optimization are discussed, including gradient-based methods, sequential model based optimization methods, methods based on hypernetworks and supernetworks.

Labworks

4 practical homeworks and 2 oral talks (optional) every term. There is also a simple questionnaire after each pair.

Grading

The overall score is a sum of scores for practical tasks, talks and the answers for the questionnaire.

Prerequisites

Statistics, linear algebra, machine learning, deep learning, intro to bayesian inference.