composer.algorithms.selective_backprop#
Modules
Core SelectiveBackprop class and functions. |
Selective Backprop prunes minibatches according to the difficulty of the individual training examples, and only computes weight gradients over the pruned subset, reducing iteration time and speeding up training.
The algorithm runs on INIT
and AFTER_DATLOADER
.
On Event.INIT, it gets the loss function before the model is wrapped. On Event.AFTER_DATALOADER, it applies selective
backprop if the time is between self.start
and self.end
.
See the Method Card for more details.
Functions
Prunes minibatches as a subroutine of SelectiveBackprop. |
|
Decides if selective backprop should be run based on time in training. |
Classes
Selectively backpropagate gradients from a subset of each batch. |