imagenet

ImageNet classification streaming dataset.

The most widely used dataset for Image Classification algorithms. Please refer to the ImageNet 2012 Classification Dataset for more details.

Classes

StreamingImageNet1k

Implementation of the ImageNet1k dataset using StreamingDataset.

class composer.datasets.imagenet.StreamingImageNet1k(remote, local, split, shuffle, resize_size=- 1, crop_size=224, batch_size=None)

Bases: composer.datasets.streaming.dataset.StreamingDataset, torchvision.datasets.vision.VisionDataset

Implementation of the ImageNet1k dataset using StreamingDataset.

Parameters

• remote (str) – Remote directory (S3 or local filesystem) where dataset is stored.

• local (str) – Local filesystem directory where dataset is cached during operation.

• split (str) – The dataset split to use, either ‘train’ or ‘val’.

• shuffle (bool) – Whether to shuffle the samples in this dataset.

• resize_size (int, optional) – The resize size to use. Use -1 to not resize. Default: -1.

• size (crop) – The crop size to use. Default: 224.

• batch_size (Optional[int]) – Hint batch_size that will be used on each device’s DataLoader. Default: None.

decode_class(data)

Decode the sample class.

Parameters

data (bytes) – The raw bytes.

Returns

np.int64 – The class encoded by the bytes.

decode_image(data)

Decode the sample image.

Parameters

data (bytes) – The raw bytes.

Returns

Image – PIL image encoded by the bytes.