utils#
Utility and helper functions for datasets.
Functions
Add a transform to a dataset's collection of transforms. |
|
Constructs a length 2 tuple of torch.Tensors from datasets that yield samples of type |
Classes
Normalizes input data and removes the background class from target data if desired. |
- class composer.datasets.utils.NormalizationFn(mean, std, ignore_background=False)[source]#
Normalizes input data and removes the background class from target data if desired.
An instance of this class can be used as the
device_transforms
argument when constructing aDataSpec
. When used here, the data will normalized after it has been loaded onto the device (i.e., GPU).- Parameters
mean (Tuple[float, float, float]) โ The mean pixel value for each channel (RGB) for the dataset.
std (Tuple[float, float, float]) โ The standard deviation pixel value for each channel (RGB) for the dataset.
ignore_background (bool) โ If
True
, ignore the background class in the training loss. Only used in semantic segmentation. Default:False
.
- composer.datasets.utils.add_vision_dataset_transform(dataset, transform, is_tensor_transform=False)[source]#
Add a transform to a datasetโs collection of transforms.
- Parameters
dataset (VisionDataset) โ A torchvision dataset.
transform (Callable) โ Function to be added to the datasetโs collection of transforms.
is_tensor_transform (bool) โ
Whether
transform
acts on data of the typeTensor
. default:False
.If
True
, andToTensor
is present in the transforms of thedataset
, thentransform
will be inserted after theToTensor
transform.If
False
andToTensor
is present, thetransform
will be inserted beforeToTensor
.If
ToTensor
is not present, the transform will be appended to the end of collection of transforms.
- Returns
None โ The
dataset
is modified in-place.
- composer.datasets.utils.pil_image_collate(batch, memory_format=torch.contiguous_format)[source]#
Constructs a length 2 tuple of torch.Tensors from datasets that yield samples of type
PIL.Image.Image
.This function can be used as the
collate_fn
argument of atorch.utils.data.DataLoader
.- Parameters
batch (List[Tuple[Image.Image, Union[Image.Image, np.ndarray]]]) โ List of (image, target) tuples that will be aggregated and converted into a single (
Tensor
,Tensor
) tuple.memory_format (memory_format) โ The memory format for the input and target tensors.
- Returns
(torch.Tensor, torch.Tensor) โ Tuple of (image tensor, target tensor) The image tensor will be four-dimensional (NCHW or NHWC, depending on the
memory_format
).