Neural Ensemble Search

Neural Ensemble Search employs Neural Architecture Search to find an optimal ensemble of models with varying architectures.

To find the optimal ensemble, NES solves a bilevel optimization problem. There are two search strategies described in the original paper:

• NES-RS (Random Search): A simple yet effective strategy relying on random uniform sampling from the search space.
• NES-RE (Regularized Evolution): Utilizes an evolutionary algorithm with tournament selection and mutations to evolve a population of strong and diverse architectures.

Sheheryar Zaidi et al."Neural Ensemble Search for Uncertainty Estimation and Dataset Shift" (2021)