cifar_hparams#

CIFAR image classification dataset hyperparameters.

The CIFAR datasets are a collection of labeled 32x32 colour images. Please refer to the CIFAR dataset for more details.

Hparams

These classes are used with yahp for YAML-based configuration.

CIFAR10DatasetHparams

Defines an instance of the CIFAR-10 dataset for image classification from a local disk.

StreamingCIFAR10Hparams

Streaming CIFAR10 hyperparameters.
class composer.datasets.cifar_hparams.CIFAR10DatasetHparams(use_synthetic=False, synthetic_num_unique_samples=100, synthetic_device='cpu', synthetic_memory_format=MemoryFormat.CONTIGUOUS_FORMAT, drop_last=True, shuffle=True, download=True, use_ffcv=False, ffcv_dir='/tmp', ffcv_dest='cifar_train.ffcv', ffcv_write_dataset=False, is_train=True, datadir=None)[source]#

Bases: composer.datasets.dataset_hparams.DatasetHparams, composer.datasets.synthetic_hparams.SyntheticHparamsMixin

Defines an instance of the CIFAR-10 dataset for image classification from a local disk.

Parameters

  • download (bool) โ€“ Whether to download the dataset, if needed. Default: True.

  • use_ffcv (bool) โ€“ Whether to use FFCV dataloaders. Default: False.

  • ffcv_dir (str) โ€“ A directory containing train/val <file>.ffcv files. If these files donโ€™t exist and ffcv_write_dataset is True, train/val <file>.ffcv files will be created in this dir. Default: "/tmp".

  • ffcv_dest (str) โ€“ <file>.ffcv file that has dataset samples. Default: "cifar_train.ffcv".

  • ffcv_write_dataset (std) โ€“ Whether to create dataset in FFCV format (<file>.ffcv) if it doesnโ€™t exist. Default:

  • False. โ€“

  • datadir (str) โ€“ The path to the data directory.

  • is_train (bool) โ€“ Whether to load the training data or validation data. Default: True.

class composer.datasets.cifar_hparams.StreamingCIFAR10Hparams(drop_last=True, shuffle=True, remote='s3://mosaicml-internal-dataset-cifar10/mds/1/', local='/tmp/mds-cache/mds-cifar10/', split='train')[source]#

Bases: composer.datasets.dataset_hparams.DatasetHparams

Streaming CIFAR10 hyperparameters.

Parameters

  • remote (str) โ€“ Remote directory (S3 or local filesystem) where dataset is stored. Default: 's3://mosaicml-internal-dataset-cifar10/mds/1/'

  • local (str) โ€“ Local filesystem directory where dataset is cached during operation. Default: '/tmp/mds-cache/mds-cifar10/'

  • split (str) โ€“ The dataset split to use, either โ€˜trainโ€™ or โ€˜valโ€™. Default: 'train'.