composer.algorithms

We describe programmatic modifications to the model or training process as “algorithms.” Examples include smoothing the labels and adding Squeeze-and-Excitation blocks, among many others.

Algorithms can be used in two ways:

  • Using Algorithm objects. These objects provide callbacks to be run in the training loop.

  • Using algorithm-specific functions and classes, such as smooth_labels or SqueezeExcite2d.

The former are the easier to compose together, since they all have the same public interface and work automatically with the Composer Trainer. The latter are easier to integrate piecemeal into an existing codebase.

See Algorithm for more information.

The following algorithms are available in Composer:

Alibi

Algorithm

Standalone

Augmix

Algorithm

Standalone

BlurPool

Algorithm

Standalone

Channels Last

Algorithm

ColOut

Algorithm

Standalone

CutOut

Algorithm

Standalone

Ghost Batch Normalization

Algorithm

Standalone

Label Smoothing

Algorithm

Standalone

Layer Freezing

Algorithm

Standalone

MixUp

Algorithm

Standalone

Progressive Resizing

Algorithm

Standalone

RandAugment

Algorithm

Standalone

Sequence Length Warmup

Algorithm

Standalone

Sharpness-Aware Minimization

Algorithm

Scaling the Learning Rate Schedule

Algorithm

Standalone

Selective Backpropagation

Algorithm

Squeeze-and-Excitation

Algorithm

Standalone

Stochastic Depth

Algorithm

Standalone

Stochastic Weight Averaging

Algorithm