The course is devoted to the problem of optimal model selection in machine learning and data analysis tasks. During the course, we investigate the Bayesian approach to model selection, its theoretical and practical aspects. We consider properties of model selection methods both specific to some notable model families or general properties suitable for the most model families.
The course investigates the properties of model selection for different types of models: from linear to mixtures of experts built on deep learning models. Various approaches to model selection and hyperparameter optimization are discussed, including gradient-based methods, sequential model based optimization methods, methods based on hypernetworks and supernetworks.
4 practical homeworks and 2 oral talks (optional) every term. There is also a simple questionnaire after each pair.
The overall score is a sum of scores for practical tasks, talks and the answers for the questionnaire.
Statistics, linear algebra, machine learning, deep learning, intro to bayesian inference.