torch_blue.vi.distributions.NonBayesian

class torch_blue.vi.distributions.NonBayesian(loss_type: str | None = None)

Bases: torch_blue.vi.distributions.base.Distribution

Pseudo-distribution that imitates non-Bayesian behavior.

This distribution is implemented as prior, variational distribution, and predictive distribution. Its distribution parameter is “mean” representing a fixed value.

As prior it functions as a uniform prior.

As variational distribution it causes fixed weights.

As predictive distribution it imitates a non-Bayesian loss, which needs to be specified during initialization.

Parameters:: loss_type (Optional[str], default = None) – Type of loss function to be used. Available options are: MAE, L1, MSE, L2
Raises:: ValueError – If loss_type is not supported.

static prior_log_prob(sample: torch.Tensor) → torch.Tensor

Compute the log probability of a sample based on the prior.

Since any sample is equally likely, the log probability for each is equal with an infinite normalization constant. Since this is hardly useful for practical use and constants may be offset during training, the log probability is always returned as zero.

This is not affected by _globals._USE_NORM_CONSTANTS.

Parameters:: sample (Tensor) – A Tensor of values to calculate the log probability for.
Returns:: The log probability of the sample under the prior, i.e. zero.
Return type:: Tensor

variational_log_prob(sample: torch.Tensor, mean: torch.Tensor) → torch.Tensor

Return 0 as dummy log probability.

Dummy log_prob that returns 0.

Parameters:

sample (Tensor) – The current weight configuration.
mean (Tensor) – The current weight values. Usually this should be the same as sample, but this is not enforce.

Returns:

A Tensor of zeroes the same shape as sample.

Return type:

Tensor

sample(mean: torch.Tensor) → torch.Tensor

Return input as sample.

Dummy sample that returns mean.

Parameters:: mean (Tensor) – The current weight values.
Returns:: The unchanged weight values.
Return type:: Tensor

static predictive_parameters_from_samples(samples: torch.Tensor) → torch.Tensor

Calculate predictive mean from samples.

Parameters:: samples (Tensor) – The model output as Tensor of shape (S, *), where S is the number of samples.
Returns:: The predictive mean as Tensor of shape (*), i.e., the average along the sample dimension.
Return type:: Tensor

log_prob_from_parameters(reference: torch.Tensor, parameters: torch.Tensor) → torch.Tensor

Calculate the loss of the mean prediction with respect to reference.

Since the loss works on NEGATIVE log likelihood this is minus the specified loss. This is not affected by _globals._USE_NORM_CONSTANTS.

Parameters:

reference (Tensor) – The ground truth label as Tensor of the same shape as parameters.
parameters (Tensor) – The predictive means as Tensor of shape as reference and as returned by predictive_parameters_from_samples().

Returns:

The loss of the prediction with respect to the reference. Shape: (1,).

Return type:

Tensor