composer.algorithms.algorithm_hparams_registry

composer.algorithms.algorithm_hparams_registry

Classes

Algorithm	Base class for algorithms.
Alibi	ALiBi (Attention with Linear Biases; Press et al, 2021) dispenses with position embeddings and instead directly biases attention matrices such that nearby tokens attend to one another more strongly.
AugMix	The AugMix data augmentation technique.
BlurPool	BlurPool adds anti-aliasing filters to convolutional layers.
ChannelsLast	Changes the memory format of the model to torch.channels_last.
ColOut	Drops a fraction of the rows and columns of an input image and (optionally) a target image.
CutMix	CutMix trains the network on non-overlapping combinations of pairs of examples and interpolated targets rather than individual examples and targets.
CutOut	CutOut is a data augmentation technique that works by masking out one or more square regions of an input image.
EMA	Maintains a shadow model with weights that follow the exponential moving average of the trained model weights.
Factorize	Decomposes linear operators into pairs of smaller linear operators.
FusedLayerNorm	Replaces all instances of torch.nn.LayerNorm with a apex.normalization.fused_layer_norm.FusedLayerNorm.
GatedLinearUnits	Replaces all instances of Linear layers in the feed-forward subnetwork with a Gated Linear Unit.
GhostBatchNorm	Replaces batch normalization modules with Ghost Batch Normalization modules that simulate the effect of using a smaller batch size.
GradientClipping	Clips all gradients in model based on specified clipping_type.
LabelSmoothing	Shrink targets towards a uniform distribution as in Szegedy et al.
LayerFreezing	Progressively freeze the layers of the network during training, starting with the earlier layers.
MixUp	MixUp trains the network on convex batch combinations.
NoOpModel	Runs on Event.INIT and replaces the model with a dummy NoOpModelClass instance.
ProgressiveResizing	Resize inputs and optionally outputs by cropping or interpolating.
RandAugment	Randomly applies a sequence of image data augmentations to an image.
SAM	Adds sharpness-aware minimization (Foret et al, 2020) by wrapping an existing optimizer with a SAMOptimizer.
SWA	Applies Stochastic Weight Averaging (Izmailov et al, 2018).
SelectiveBackprop	Selectively backpropagate gradients from a subset of each batch.
SeqLengthWarmup	Progressively increases the sequence length during training.
SqueezeExcite	Adds Squeeze-and-Excitation blocks (Hu et al, 2019) after the torch.nn.Conv2d modules in a neural network.
StochasticDepth	Applies Stochastic Depth (Huang et al, 2016) to the specified model.

Attributes

• Dict

• Type

• Union

• algorithm_registry