composer.algorithms.functional.apply_ghost_batchnorm
- composer.algorithms.functional.apply_ghost_batchnorm(model: torch.nn.modules.module.Module, ghost_batch_size: int) torch.nn.modules.module.Module[source]
Replace batch normalization modules with ghost batch normalization modules.
Must be run before the model has been moved to accelerators and before the model’s parameters have been passed to an optimizer.
- Parameters
model – model to transform
ghost_batch_size – size of sub-batches to normalize over