Research notes, library releases, and other updates.
MOTPE extends the Tree-Structured Parzen Estimator (TPE) to multiobjective optimization, handling complex, conditional search spaces with high efficiency. It uses dominance-based splitting and hype...
A blog post about robust regression with Student-t processes and efficient O(n) inference via state-space models.
Casting Multi-Task Learning as Multi-Objective Optimization using a scalable gradient-based algorithm (MGDA-UB) to find Pareto optimal solutions in deep networks.
SC-VAMP: a Jacobian-free variant of Vector Approximate Message Passing that replaces expensive divergence computations with score-function norms, enabling Bayes-optimal inference with deep neural d...
A gentle blog-style guide to EnVI and OEnVI for Gaussian process state-space models. Based on the paper 'Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference...
Malach et al. (2020) prove a strong form of the Lottery Ticket Hypothesis: every sufficiently overparameterized randomly-initialized neural network contains a subnetwork that, without any training,...
Forward-mode Differentiation with hyperparameter Sharing (FDS), a gradient-based hyperparameter optimization method.
Hyperband is a novel bandit-based approach to hyperparameter optimization that speeds up random search by adaptive resource allocation and early-stopping. It outperforms Bayesian optimization metho...
Deep image classifiers’ logits retain far more visual information than just class labels, enabling realistic image reconstruction via a GAN and revealing model invariances, robustness, and sensitiv...
Spherical loss functions and their performance compared to log-softmax in multi-class classification tasks.