stochastic_layers#

Stochastic forward functions for ResNet Bottleneck modules.

Functions

make_resnet_bottleneck_stochastic

Model surgery policy that dictates how to convert a ResNet Bottleneck layer into a stochastic version.

Classes

BlockStochasticModule

A convenience class that stochastically executes the provided main path of a residual block.
class composer.algorithms.stochastic_depth.stochastic_layers.BlockStochasticModule(main, residual=None, drop_rate=0.2)[source]#

Bases: torch.nn.modules.module.Module

A convenience class that stochastically executes the provided main path of a residual block.

Parameters

  • main (GraphModule) โ€“ Operators in the main (non-residual) path of a residual block.

  • residual (GraphModule | None) โ€“ Operators, if any, in the residual path of a residual block.

  • drop_rate โ€“ The base probability of dropping this layer. Must be between 0.0 (inclusive) and 1.0 (inclusive).

Returns

BlockStochasticModule โ€“ An instance of BlockStochasticModule.

composer.algorithms.stochastic_depth.stochastic_layers.block_stochastic_forward(self, x)[source]#

ResNet Bottleneck forward function where the layers are randomly skipped with probability drop_rate during training.

composer.algorithms.stochastic_depth.stochastic_layers.make_resnet_bottleneck_stochastic(module, module_index, module_count, drop_rate, drop_distribution, stochastic_method)[source]#

Model surgery policy that dictates how to convert a ResNet Bottleneck layer into a stochastic version.

composer.algorithms.stochastic_depth.stochastic_layers.sample_stochastic_forward(self, x)[source]#

ResNet Bottleneck forward function where samples are randomly dropped with probability drop_rate during training.