composer.datasets.imagenet#
composer.datasets.imagenet
Functions
Returns the same class as was passed in, with dunder methods added based on the fields defined in the class. |
|
Constructs a |
Classes
Specifications for operating and training on data. |
|
|
A generic data loader where the images are arranged in this way by default: . |
Normalizes input data and removes the background class from target data if desired. |
|
Emulates a dataset of provided size and shape. |
Hparams
These classes are used with yahp
for YAML
-based configuration.
Hyperparameters to initialize a |
|
Abstract base class for hyperparameters to initialize a dataset. |
|
Defines an instance of the ImageNet dataset for image classification. |
|
Synthetic dataset parameter mixin for |
Attributes
IMAGENET_CHANNEL_MEAN
IMAGENET_CHANNEL_STD
List
- class composer.datasets.imagenet.ImagenetDatasetHparams(use_synthetic=False, synthetic_num_unique_samples=100, synthetic_device='cpu', synthetic_memory_format=MemoryFormat.CONTIGUOUS_FORMAT, is_train=True, drop_last=True, shuffle=True, datadir=None, resize_size=- 1, crop_size=224)[source]#
Bases:
composer.datasets.hparams.DatasetHparams
,composer.datasets.hparams.SyntheticHparamsMixin
Defines an instance of the ImageNet dataset for image classification.
- Parameters
use_synthetic (bool, optional) โ Whether to use synthetic data. (Default:
False
)synthetic_num_unique_samples (int, optional) โ The number of unique samples to allocate memory for. Ignored if
use_synthetic
is False. (Default:100
)synthetic_device (str, optonal) โ The device to store the sample pool. Set to
cuda
to store samples on the GPU and eliminate PCI-e bandwidth with the dataloader. Set tocpu
to move data between host memory and the device on every batch. Ignored ifuse_synthetic
is False. (Default:cpu
)synthetic_memory_format โ The
MemoryFormat
to use. Ignored ifuse_synthetic
is False. (Default:CONTIGUOUS_FORMAT
)datadir (str) โ The path to the data directory.
is_train (bool) โ Whether to load the training data (the default) or validation data.
drop_last (bool) โ If the number of samples is not divisible by the batch size, whether to drop the last batch (the default) or pad the last batch with zeros.
shuffle (bool) โ Whether to shuffle the dataset. Defaults to True.
resize_size (int, optional) โ The resize size to use. Defaults to -1 to not resize.
size (crop) โ The crop size to use.
- initialize_object(batch_size, dataloader_hparams)[source]#
Creates a
DataLoader
orDataloaderSpec
for this dataset.- Parameters
batch_size (int) โ The size of the batch the dataloader should yield. This batch size is device-specific and already incorporates the world size.
dataloader_hparams (DataloaderHparams) โ The dataset-independent hparams for the dataloader
- Returns
Dataloader or DataSpec โ The dataloader, or if the dataloader yields batches of custom types,
a :class:`DataSpec`.