composer.datasets.cifar10#

composer.datasets.cifar10

Functions

dataclass

Returns the same class as was passed in, with dunder methods added based on the fields defined in the class.

Classes

CIFAR10

CIFAR10 Dataset.

DataLoader

Protocol for custom DataLoaders compatible with torch.utils.data.DataLoader.

SyntheticBatchPairDataset

Emulates a dataset of provided size and shape.

Hparams

These classes are used with yahp for YAML-based configuration.

CIFAR10DatasetHparams

Defines an instance of the CIFAR-10 dataset for image classification.

DataloaderHparams

Hyperparameters to initialize a Dataloader.

DatasetHparams

Abstract base class for hyperparameters to initialize a dataset.

SyntheticHparamsMixin

Synthetic dataset parameter mixin for DatasetHparams.
class composer.datasets.cifar10.CIFAR10DatasetHparams(use_synthetic=False, synthetic_num_unique_samples=100, synthetic_device='cpu', synthetic_memory_format=MemoryFormat.CONTIGUOUS_FORMAT, is_train=True, drop_last=True, shuffle=True, datadir=None, download=True)[source]#

Bases: composer.datasets.hparams.DatasetHparams, composer.datasets.hparams.SyntheticHparamsMixin

Defines an instance of the CIFAR-10 dataset for image classification.

Parameters

  • use_synthetic (bool, optional) – Whether to use synthetic data. (Default: False)

  • synthetic_num_unique_samples (int, optional) – The number of unique samples to allocate memory for. Ignored if use_synthetic is False. (Default: 100)

  • synthetic_device (str, optonal) – The device to store the sample pool. Set to cuda to store samples on the GPU and eliminate PCI-e bandwidth with the dataloader. Set to cpu to move data between host memory and the device on every batch. Ignored if use_synthetic is False. (Default: cpu)

  • synthetic_memory_format – The MemoryFormat to use. Ignored if use_synthetic is False. (Default: CONTIGUOUS_FORMAT)

  • datadir (str) – The path to the data directory.

  • is_train (bool) – Whether to load the training data (the default) or validation data.

  • drop_last (bool) – If the number of samples is not divisible by the batch size, whether to drop the last batch (the default) or pad the last batch with zeros.

  • shuffle (bool) – Whether to shuffle the dataset. Defaults to True.

  • download (bool) – Whether to download the dataset, if needed.

initialize_object(batch_size, dataloader_hparams)[source]#

Creates a DataLoader or DataloaderSpec for this dataset.

Parameters

  • batch_size (int) – The size of the batch the dataloader should yield. This batch size is device-specific and already incorporates the world size.

  • dataloader_hparams (DataloaderHparams) – The dataset-independent hparams for the dataloader

Returns

  • Dataloader or DataSpec – The dataloader, or if the dataloader yields batches of custom types,

  • a :class:`DataSpec`.