composer.datasets.evaluator#

composer.datasets.evaluator

Functions

`dataclass`	Returns the same class as was passed in, with dunder methods added based on the fields defined in the class.
`get_dataset_registry`	composer.datasets.dataset_registry.get_dataset_registry

Classes

`ComposerModel`	The minimal interface needed to use a model with `composer.trainer.Trainer`.
`Evaluator`	Wrapper for a dataloader to include metrics that apply to a specific dataset.
`Metric`	Base class for all metrics present in the Metrics API.
`MetricCollection`	MetricCollection class can be used to chain metrics that have the same call pattern into one single class.

Hparams

These classes are used with yahp for YAML-based configuration.

`DataloaderHparams`	Hyperparameters to initialize a `Dataloader`.
`DatasetHparams`	Abstract base class for hyperparameters to initialize a dataset.
`EvaluatorHparams`	Params for the `Evaluator`.

Attributes

class composer.datasets.evaluator.EvaluatorHparams(label, eval_dataset, metric_names=None)[source]#

Params for the Evaluator.

See the documentation for the Evaluator.

initialize_object(model, batch_size, dataloader_hparams)[source]#

Initialize an Evaluator

If the Evaluatormetric_names is empty or None is provided, the function returns a copy of all the model’s default evaluation metrics.

Parameters

model (ComposerModel) – The model, which is used to retrieve metric names
batch_size (int) – The device batch size to use for the evaluation dataset
dataloader_hparams (DataloaderHparams) – The hparams to use to construct a dataloader for the evaluation dataset

Returns

Evaluator – The evaluator