composer.datasets

Modules

composer.datasets.ade20k	composer.datasets.ade20k
composer.datasets.brats	composer.datasets.brats
composer.datasets.cifar10	composer.datasets.cifar10
composer.datasets.dataloader	composer.datasets.dataloader
composer.datasets.dataset_registry	composer.datasets.dataset_registry
composer.datasets.evaluator	composer.datasets.evaluator
composer.datasets.glue	composer.datasets.glue
composer.datasets.hparams	composer.datasets.hparams
composer.datasets.imagenet	composer.datasets.imagenet
composer.datasets.lm_datasets	composer.datasets.lm_datasets
composer.datasets.mnist	composer.datasets.mnist
composer.datasets.synthetic	composer.datasets.synthetic
composer.datasets.utils	composer.datasets.utils

Datasets TODO – more description.

DataloaderHparams contains the torch.utils.data.dataloader settings that are common across both training and eval datasets:

• num_workers

• prefetch_factor

• persistent_workers

• pin_memory

• timeout

Each DatasetHparams is then responsible for settings such as:

• dataset

• drop_last

• shuffle

• collate_fn

A DatasetHparams is responsible for returning a torch.utils.data.dataloader or a DataloaderSpec.

Functions

get_dataset_registry

composer.datasets.dataset_registry.get_dataset_registry

Classes

MemoryFormat	An enumeration.
SyntheticBatchPairDataset	Emulates a dataset of provided size and shape.
SyntheticDataLabelType	An enumeration.
SyntheticDataType	An enumeration.
SyntheticPILDataset	Similar to SyntheticBatchPairDataset, but yields samples of type Image and supports dataset transformations.
WrappedDataLoader	composer.datasets.dataloader.WrappedDataLoader

Hparams

These classes are used with yahp for YAML-based configuration.

ADE20kDatasetHparams	Defines an instance of the ADE20k dataset for semantic segmentation.
BratsDatasetHparams	Defines an instance of the BraTS dataset for image segmentation.
CIFAR10DatasetHparams	Defines an instance of the CIFAR-10 dataset for image classification.
DataloaderHparams	Hyperparameters to initialize a Dataloader.
DatasetHparams	Abstract base class for hyperparameters to initialize a dataset.
EvaluatorHparams	Params for the Evaluator.
GLUEHparams	Sets up a generic GLUE dataset loader.
ImagenetDatasetHparams	Defines an instance of the ImageNet dataset for image classification.
LMDatasetHparams	Defines a generic dataset class for autoregressive and masked language models trained with self-supervised learning.
MNISTDatasetHparams	Defines an instance of the MNIST dataset for image classification.
SyntheticHparamsMixin	Synthetic dataset parameter mixin for DatasetHparams.