composer.datasets.evaluator_hparams#
Hyperparameters for the Evaluator
.
Hparams
These classes are used with yahp
for YAML
-based configuration.
Params for the |
- class composer.datasets.evaluator_hparams.EvaluatorHparams(label, eval_dataset, eval_interval=None, subset_num_batches=None, metric_names=None)[source]#
Bases:
yahp.hparams.Hparams
Params for the
Evaluator
.Also see the documentation for the
Evaluator
.- Parameters
label (str) โ Name of the Evaluator. Used for logging/reporting metrics.
eval_dataset (DatasetHparams) โ Evaluation dataset.
metrics (list, optional) โ List of strings of names of the metrics for the evaluator. Can be a
torchmetrics.Metric
name or the class name of a metric returned bymetrics()
IfNone
, uses all metrics in the model. Default:None
.
- initialize_object(model, batch_size, dataloader_hparams)[source]#
Initialize an
Evaluator
If the Evaluator
metric_names
is empty or None is provided, the function returns a copy of all the modelโs default evaluation metrics.- Parameters
model (ComposerModel) โ The model, which is used to retrieve metric names.
batch_size (int) โ The device batch size to use for the evaluation dataset.
dataloader_hparams (DataLoaderHparams) โ The hparams to use to construct a dataloader for the evaluation dataset.
- Returns
Evaluator โ The evaluator.