Losses
Objectives for training variational Bayesian models.
Variational Loss
For alpha=1, pass the analytic KL as the third argument. For alpha != 1, generate matched draws with bensemble.utils.sample_with_log_probs and pass log_prior and log_posterior as keyword arguments. A scalar KL alone raises ValueError on the Rényi path. See the training example and migration notes.
bensemble.losses.VariationalLoss
VariationalLoss(likelihood_model: Module, alpha: float = 1.0, num_batches: int = 1, kl_weight: float = 1.0)
Bases: Module
Variational loss supporting ELBO and Rényi alpha-divergence.
Initializes the VariationalLoss module.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
likelihood_model | Module | Module returning element-wise or summed negative log-likelihood. Use reduction="none" or "sum", not "mean". | required |
alpha | float | Rényi parameter. At 1.0, use the ELBO. Other values require sampled log prior and posterior densities in forward(). | 1.0 |
num_batches | int | Number of equal-sized training batches. The Rényi path estimates the full-data log likelihood, then divides the final objective by this number. Defaults to 1 (full-batch training). | 1 |
kl_weight | float | Weight on the prior/posterior log ratio or analytic KL. The standard variational objective uses 1.0. | 1.0 |
Source code in bensemble/losses.py
forward
forward(preds: Tensor, target: Tensor, kl_divergence: Tensor | None = None, *, log_prior: Tensor | None = None, log_posterior: Tensor | None = None) -> torch.Tensor
Computes variational objective loss.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
preds | Tensor | Predictions tensor of shape (K, batch_size, ...) or (batch_size, ...). | required |
target | Tensor | Target tensor of shape (batch_size, ...). | required |
kl_divergence | Tensor | None | Analytic KL, supported only for alpha=1.0. Omit when providing sampled log densities. | None |
log_prior | Tensor | None | Log prior density of each weight draw, shape (K,). | None |
log_posterior | Tensor | None | Log posterior density of the same draws, shape (K,). Each pair must describe the weights used for preds[k]. Use bensemble.utils.sample_with_log_probs to generate matched draws. | None |
Returns:
| Type | Description |
|---|---|
Tensor | torch.Tensor: Scalar loss value. |
Source code in bensemble/losses.py
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 | |
Gaussian Likelihood
bensemble.losses.GaussianLikelihood
Bases: Module
Gaussian Likelihood with learnable homoscedastic uncertainty.
Learns a global standard deviation (sigma) for data noise and computes negative log-likelihood via Gaussian NLL.
Initializes the GaussianLikelihood layer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
init_log_sigma | float | Initial value for log standard deviation. Defaults to -2.0. | -2.0 |
Source code in bensemble/losses.py
forward
Computes Gaussian negative log-likelihood loss.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
preds | Tensor | Predicted tensor of shape (*). | required |
target | Tensor | Target tensor of shape (*). | required |
Returns:
| Type | Description |
|---|---|
Tensor | torch.Tensor: Element-wise negative log-likelihood tensor matching target shape. |