Skip to content

Experiment Group#

ExperimentGroup #

This class represents a group of related Experiments.

The results across experiments can be aggregated to give an average result for the group.

For example, this could represent the same model evaluated across different folds of the same dataset. Or they could be results on te same dataset from models with different weight initializations.

Parameters:

  • name (str) –

    the name of this experiment group

  • rng (RNG) –

    the RNG used to control randomness

Attributes#

num_experiments property #

The number of experiments in this ExperimentGroup.

Configuration#

add_experiment #

Adds an Experiment to this ExperimentGroup.

Each experiment is characterized by a single confusion matrix.

Parameters:

  • name (str) –

    the name of this experiment

  • confusion_matrix (Int[ndarray, 'num_classes num_classes']) –

    the confusion matrix for this experiment.

  • prevalence_prior (str | float | Float[ArrayLike, ' num_classes'], default: 0 ) –

    the prior over the prevalence counts for this experiments. Defaults to 0, Haldane's prior.

  • confusion_prior (str | float | Float[ArrayLike, ' num_classes num_classes'], default: 0 ) –

    the prior over the confusion counts for this experiments. Defaults to 0, Haldane's prior.

__getitem__ #

Gets an experiment under this ExperimentGroup by its name.

Parameters:

  • key (str) –

    the experiment name

Returns:

ExperimentGroupResult dataclass #

A wrapper class for the output of an ExperimentGroup.