composer.algorithms.fused_layernorm#
Modules
composer.algorithms.fused_layernorm.fused_layernorm |
Replaces all instances of torch.nn.LayerNorm with a apex.normalization.fused_layer_norm.FusedLayerNorm.
By fusing multiple kernel launches into one, this usually improves GPU utilization.
See the Method Card for more details.
Functions
Replaces all instances of torch.nn.LayerNorm with a apex.normalization.fused_layer_norm.FusedLayerNorm. |
Classes
Replaces all instances of torch.nn.LayerNorm with a apex.normalization.fused_layer_norm.FusedLayerNorm. |