composer.algorithms.label_smoothing.label_smoothing#

Core Label Smoothing classes and functions.

Functions

smooth_labels

Shrinks targets towards a uniform distribution to counteract label noise as in Szegedy et al.

Classes

LabelSmoothing

Shrinks targets towards a uniform distribution to counteract label noise as in Szegedy et al.
class composer.algorithms.label_smoothing.label_smoothing.LabelSmoothing(alpha)[source]#

Bases: composer.core.algorithm.Algorithm

Shrinks targets towards a uniform distribution to counteract label noise as in Szegedy et al.

This is computed by (1 - alpha) * targets + alpha * smoothed_targets where smoothed_targets is a vector of ones.

Introduced in Rethinking the Inception Architecture for Computer Vision.

Example

from composer.algorithms import LabelSmoothing
from composer.trainer import Trainer
label_smoothing_algorithm = LabelSmoothing(alpha=0.1)
trainer = Trainer(
    model=model,
    train_dataloader=train_dataloader,
    eval_dataloader=eval_dataloader,
    max_duration="1ep",
    algorithms=[label_smoothing_algorithm],
    optimizers=[optimizer]
)
Parameters

alpha – Strength of the label smoothing, in [0, 1]. alpha=0 means no label smoothing, and alpha=1 means maximal smoothing (targets are ignored).

composer.algorithms.label_smoothing.label_smoothing.smooth_labels(logits, targets, alpha)[source]#

Shrinks targets towards a uniform distribution to counteract label noise as in Szegedy et al.

This is computed by (1 - alpha) * targets + alpha * smoothed_targets where smoothed_targets is a uniform distribution.

Example

from composer.algorithms.label_smoothing import smooth_labels
new_targets = smooth_labels(
                logits=logits,
                targets=y_example,
                alpha=0.1
              )
Parameters

  • logits – Output of the model. Tensor of shape (N, C, d1, …, dn) for N examples and C classes, and d1, …, dn extra dimensions.

  • targets – Tensor of shape (N) containing integers 0 <= i <= C-1 specifying the target labels for each example.

  • alpha – Strength of the label smoothing, in [0, 1]. alpha=0 means no label smoothing, and alpha=1 means maximal smoothing (targets are ignored).