composer.datasets.utils#

Utility and helper functions for datasets.

Functions

add_vision_dataset_transform

Add a transform to a dataset's collection of transforms.

pil_image_collate

Constructs a length 2 tuple of torch.Tensors from datasets that yield samples of type PIL.Image.Image.

Classes

NormalizationFn

Normalizes input data and removes the background class from target data if desired.
class composer.datasets.utils.NormalizationFn(mean, std, ignore_background=False)[source]#

Normalizes input data and removes the background class from target data if desired.

An instance of this class can be used as the device_transforms argument when constructing a DataSpec. When used here, the data will normalized after it has been loaded onto the device (i.e., GPU).

Parameters

  • mean (Tuple[float, float, float]) โ€“ The mean pixel value for each channel (RGB) for the dataset.

  • std (Tuple[float, float, float]) โ€“ The standard deviation pixel value for each channel (RGB) for the dataset.

  • ignore_background (bool) โ€“ If True, ignore the background class in the training loss. Only used in semantic segmentation. Default: False.

composer.datasets.utils.add_vision_dataset_transform(dataset, transform, is_tensor_transform=False)[source]#

Add a transform to a datasetโ€™s collection of transforms.

Parameters

  • dataset (VisionDataset) โ€“ A torchvision dataset.

  • transform (Callable) โ€“ Function to be added to the datasetโ€™s collection of transforms.

  • is_tensor_transform (bool) โ€“

    Whether transform acts on data of the type Tensor. default: False.

    • If True, and ToTensor is present in the transforms of the dataset, then transform will be inserted after the ToTensor transform.

    • If False and ToTensor is present, the transform will be inserted before ToTensor.

    • If ToTensor is not present, the transform will be appended to the end of collection of transforms.

Returns

None โ€“ The dataset is modified in-place.

composer.datasets.utils.pil_image_collate(batch, memory_format=torch.contiguous_format)[source]#

Constructs a length 2 tuple of torch.Tensors from datasets that yield samples of type PIL.Image.Image.

This function can be used as the collate_fn argument of a torch.utils.data.DataLoader.

Parameters

  • batch (List[Tuple[Image.Image, Union[Image.Image, np.ndarray]]]) โ€“ List of (image, target) tuples that will be aggregated and converted into a single (Tensor, Tensor) tuple.

  • memory_format (memory_format) โ€“ The memory format for the input and target tensors.

Returns

(torch.Tensor, torch.Tensor) โ€“ Tuple of (image tensor, target tensor) The image tensor will be four-dimensional (NCHW or NHWC, depending on the memory_format).