Utilities
Helpers for stochastic inference, model pruning and Gaussian moment propagation.
sample_with_log_probs returns predictions and matching prior/posterior log densities for Rényi training. See the Rényi guide for usage; predict_with_uncertainty returns predictive mean and standard deviation and does not supply weight densities.
bensemble.utils
enable_dropout
Enables dropout layers during evaluation for Monte Carlo Dropout.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | Module | Target neural network containing PyTorch dropout layers. | required |
Source code in bensemble/utils.py
get_total_kl
Calculates the sum of KL divergences of all Bayesian layers in the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | Module | Neural network model containing Bayesian layers. | required |
Returns:
| Type | Description |
|---|---|
Tensor | torch.Tensor: Total accumulated KL divergence scalar. |
Source code in bensemble/utils.py
predict_with_uncertainty
predict_with_uncertainty(model: Module, x: Tensor, num_samples: int = 100) -> tuple[torch.Tensor, torch.Tensor]
Estimates predictive mean and standard deviation via Monte Carlo sampling.
Restores each layer's original training flag, including on errors.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | Module | Neural network model containing stochastic layers. | required |
x | Tensor | Input tensor. | required |
num_samples | int | Number of forward Monte Carlo passes. Defaults to 100. | 100 |
Returns:
| Type | Description |
|---|---|
Tensor | tuple[torch.Tensor, torch.Tensor]: A tuple (mean, std) representing |
Tensor | the predictive mean and unbiased sample standard deviation. |
Source code in bensemble/utils.py
prune_model
Applies Graves' SNR-based weight pruning to all Bayesian layers in the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | Module | Neural network containing BaseBayesianLayer modules. | required |
threshold | float | Signal-to-Noise Ratio (SNR) pruning threshold. Defaults to 0.83. | 0.83 |
Returns:
| Name | Type | Description |
|---|---|---|
float | float | Overall sparsity ratio of pruned weights across all Bayesian layers (0.0 to 1.0). |
Source code in bensemble/utils.py
sample_with_log_probs
sample_with_log_probs(model: Module, x: Tensor, num_samples: int = 10) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]
Draws model weights and returns predictions with their log densities.
Each draw uses one set of weights across the batch and every call to a reused layer. Unlike local reparameterization, these predictions have matching parameter-space prior and posterior densities for a Rényi objective. Model parameters, buffers, and training modes are preserved; BatchNorm running statistics are not updated.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model | Module | Model containing BayesianLinear or BayesianConv2d layers. Other layers retain their training modes; set stochastic layers such as dropout to eval if only Bayesian weight uncertainty is wanted. | required |
x | Tensor | Input batch. | required |
num_samples | int | Positive number of weight draws. | 10 |
Returns:
| Type | Description |
|---|---|
Tensor | Tuple of predictions shaped (K, batch, ...), log prior densities (K,), |
Tensor | and log posterior densities (K,). All three retain gradients. |
Raises:
| Type | Description |
|---|---|
ValueError | If the sample count is invalid, no supported layers exist, or distinct layers share posterior means. |
TypeError | If the model contains an unsupported Bayesian layer. |
Source code in bensemble/utils.py
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 | |
standard_normal_cdf
Computes the standard normal cumulative distribution function Phi(x).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x | Tensor | Input tensor. | required |
Returns:
| Type | Description |
|---|---|
Tensor | torch.Tensor: Cumulative distribution value evaluated at x. |
Source code in bensemble/utils.py
standard_normal_pdf
Computes the standard normal probability density function phi(x).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x | Tensor | Input tensor. | required |
Returns:
| Type | Description |
|---|---|
Tensor | torch.Tensor: Probability density evaluated at x. |