composer.algorithms.label_smoothing.label_smoothing#
Core Label Smoothing classes and functions.
Functions
Shrink targets towards a uniform distribution as in Szegedy et al. |
Classes
Shrink targets towards a uniform distribution as in Szegedy et al. |
- class composer.algorithms.label_smoothing.label_smoothing.LabelSmoothing(smoothing=0.1)[source]#
Bases:
composer.core.algorithm.Algorithm
Shrink targets towards a uniform distribution as in Szegedy et al.
The smoothed labels are computed as
(1 - smoothing) * targets + smoothing * unif
whereunif
is a vector with elements all equal to1 / num_classes
.- Parameters
smoothing โ Strength of the label smoothing, in \([0, 1]\).
smoothing=0
means no label smoothing, andsmoothing=1
means maximal smoothing (targets are ignored). Default:0.1
.
Example
from composer.algorithms import LabelSmoothing algorithm = LabelSmoothing(smoothing=0.1) trainer = Trainer( model=model, train_dataloader=train_dataloader, eval_dataloader=eval_dataloader, max_duration="1ep", algorithms=[algorithm], optimizers=[optimizer] )
- composer.algorithms.label_smoothing.label_smoothing.smooth_labels(logits, target, smoothing=0.1)[source]#
Shrink targets towards a uniform distribution as in Szegedy et al.
The smoothed labels are computed as
(1 - smoothing) * targets + smoothing * unif
whereunif
is a vector with elements all equal to1 / num_classes
.- Parameters
logits (Tensor) โ predicted value for
target
, or any other tensor with the same shape. Shape must be(N, num_classes, ...)
forN
examples andnum_classes
classes, with any number of optional extra dimensions.target (Tensor) โ target tensor of either shape
N
or(N, num_classes, ...)
. In the former case, elements oftargets
must be integer class ids in the range0..num_classes
. In the latter case,targets
must have the same shape aslogits
.smoothing (float, optional) โ strength of the label smoothing, in \([0, 1]\).
smoothing=0
means no label smoothing, andsmoothing=1
means maximal smoothing (targets are ignored). Default:0.1
.
- Returns
targets_smooth (torch.Tensor) โ The smoothed targets
Example
import torch num_classes = 10 targets = torch.randint(num_classes, size=(100,)) from composer.algorithms.label_smoothing import smooth_labels new_targets = smooth_labels(logits=logits, target=targets, smoothing=0.1)