composer.algorithms.selective_backprop

Modules

composer.algorithms.selective_backprop.selective_backprop

Core SelectiveBackprop class and functions.

Selective Backprop prunes minibatches according to the difficulty of the individual training examples, and only computes weight gradients over the pruned subset, reducing iteration time and speeding up training.

The algorithm runs on INIT and AFTER_DATLOADER. On Event.INIT, it gets the loss function before the model is wrapped. On Event.AFTER_DATALOADER, it applies selective backprop if the time is between self.start and self.end.

See the Method Card for more details.

Functions

select_using_loss	Prunes minibatches as a subroutine of SelectiveBackprop.
should_selective_backprop	Decides if selective backprop should be run based on time in training.

Classes

SelectiveBackprop

Selectively backpropagate gradients from a subset of each batch.