About

The course is devoted to modern methods of constructing complex neural network architectures and choosing optimal structures.

Syllabus

• Overview of neural network types and architecture descriptions.
• Genetic algorithms from GMDH to WANN.
• Quality criteria for selecting structures (to be discussed in a couple of weeks).
• A priori assumptions about the model structure, structural parameters distribution.
• Methods for optimization of structural parameters.
• Online training and multi-armed bandits to architecture search.
• Reinforcement learning for architecture search.
• Knowledge transfer between neural networks and optimization of structural parameters.
• Random processes for architecture search.
• Generative adversarial networks and the architecture search.
• Generation and rejection of structures.
• Two-level Bayesian selection and Metropolis-Hastings sampling.

Самостоятельная работа

The laboratory work consists in the study of the architecture search method. The first job is to analyze a ready-made method, the second job is to propose and program your own method. The work report is a page of text with a formal description of the method with sufficient detail for code recovery and error analysis (basic criteria complexity, stability, accuracy). The interface to the class is fixed and common to everyone, as are the selections. There is a general table with the results, and a private error analysis of each method.

Оценивание

A total of 10 points, two points for answering questions during lectures, four points each for two laboratory work. It is not the accuracy of the approximation that is evaluated, but the quality of the code and error analysis.

Требуемые знания

Machine learning, deep learning, Bayesian model selection.