Gradient-based Hyperparameter Optimization Over Long Horizons
Forward-mode Differentiation with hyperparameter Sharing (FDS), a gradient-based hyperparameter optimization method.
Blog Posts
Research notes, library releases, and other updates.
Hyperband: Accelerating Hyperparameter Optimization via Adaptive Resource Allocation
Hyperband is a novel bandit-based approach to hyperparameter optimization that speeds up random search by adaptive resource allocation and early-stopping. It outperforms Bayesian optimization metho...
Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers
Deep image classifiers’ logits retain far more visual information than just class labels, enabling realistic image reconstruction via a GAN and revealing model invariances, robustness, and sensitiv...
An Exploration of Softmax Alternatives Belonging to the Spherical Loss Family
Spherical loss functions and their performance compared to log-softmax in multi-class classification tasks.