Getting Started
Installation
Docker
Verification
Using Composer
Functional
MosaicML Trainer
Welcome Tour
Our First Method!
A Simple Instrumented Trainer
Introducing… Events, Engines, and State
Next: The MosaicML Trainer
Custom Models/datasets
Models
Datasets
Trainer init
Trainer with YAHP
core
composer.Algorithm
composer.Callback
Callback
RankZeroCallback
composer.Engine
Trace
composer.Event
Events List
Training Loop
API Reference
composer.Logger
BaseLoggerBackendHparams
BaseLoggerBackend
RankZeroLoggerBackend
composer.State
composer.core.surgery
composer.core.types
Tensor Types
Batch Types
Dataset and Data Loader Types
Trainer Types
Miscellaneous Types
composer
composer.algorithms
Alibi
Augmix
BlurPool
Channels Last
ColOut
CutOut
Ghost Batch Normalization
Label Smoothing
Layer Freezing
MixUp
Progressive Resizing
RandAugment
Sequence Length Warmup
Sharpness-Aware Minimization
Scaling the Learning Rate Schedule
Selective Backpropagation
Squeeze-and-Excitation
Stochastic Depth
Stochastic Weight Averaging
composer.callbacks
Callbacks
Callback Hyperparameters
composer.datasets
Base Classes and Hyperparameters
Datasets
composer.functional
composer.algorithms.functional.augment_and_mix
composer.algorithms.functional.apply_blurpool
composer.algorithms.functional.colout
composer.algorithms.functional.cutout
composer.algorithms.functional.smooth_labels
composer.algorithms.functional.freeze_layers
composer.algorithms.functional.mixup_batch
composer.algorithms.functional.resize_inputs
composer.algorithms.functional.randaugment
composer.algorithms.functional.scale_scheduler
composer.algorithms.functional.selective_backprop
composer.algorithms.functional.apply_se
composer.loggers
Logger and LogLevel
Backends
Backend Hyperparameters
composer.models
Base Models
Image Models
Language Models
Metrics and Loss Functions
composer.optim
DecoupledSGDW
DecoupledAdamW
composer.optim.scheduler
WarmUpLR
ConstantLR
ComposedScheduler
composer.trainer
Examples
Trainer Hparams
API Reference
composer.trainer.devices
Devices
Device Dataloaders
Methods Library
ALiBi
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
AugMix
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
BlurPool
TL;DR
Attribution
Code and Hyperparameters
Applicable Settings
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Acknowledgments
Code
Channels Last
TLDR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Considerations
Composability
Code
ColOut
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Cutout
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Decoupled Weight Decay
TL;DR
Attribution
Code and Hyperparameters
Applicable Settings
Implementation Details
Considerations
Composability
Ghost BatchNorm
Attribution
Hyperparameters
Applicable Settings
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Label Smoothing
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Layer Freezing
TL;DR
Attribution
Hyperparameters
Applicable Settings
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
MixUp
TL;DR
Attribution
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Progressive Image Resizing
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Detailed Results
Code
RandAugment
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Sharpness Aware Minimization
TL;DR
Attribution
Code and Hyperparameters
Applicable Settings
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Scale Schedule
TL;DR
Attribution
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Composability
Code
Scaling Laws
TLDR
Attribution
Applicable Settings
Hyperparameters
Detailed Results
Considerations
Effects & Implications
Selective Backprop
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Detailed Results
Code
Squeeze-and-Excitation
TL;DR
Attribution
Code and Hyperparameters
Applicable Settings
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Stochastic Depth (Block-Wise)
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Stochastic Depth (Sample-Wise)
TL;DR
Attribution
Hyperparameters
Applicable Settings
Example Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composability
Code
Stochastic Weight Averaging
TL;DR
Attribution
Applicable Settings
Hyperparameters
Example Effects
Implementation Details
Considerations
Composability
Sequence Length Warmup
tl;dr
Hyperparameters
Applicable Settings
Effects
Implementation Details
Suggested Hyperparameters
Considerations
Composition
Attribution
Code
Model Library
CIFAR ResNet
Overview
Attribution
Architecture
Family members
Default Training Hyperparameters
EfficientNet
Overview
Attribution
Architecture
Family members
Default Training Hyperparameters
GPT-2
Overview
Attribution
Architecture
Family Members
Implementation Details
Exploring Tradeoffs Between Quality and Training Speed / Cost
ImageNet ResNet
Overview
Attribution
Architecture
Family Members
Implementation details
Default Training Hyperparameters
UNet
Overview
Attribution
Architecture
Implementation Details
Exploring Tradeoffs Between Quality and Training Speed/Cost
MosaicML
»
Index
Edit on GitHub
Index
A
|
B
|
C
|
D
|
E
|
F
|
G
|
I
|
J
|
L
|
M
|
N
|
O
|
P
|
R
|
S
|
T
|
U
|
V
|
W
A
after_train_batch() (composer.callbacks.grad_monitor.GradMonitor method)
Algorithm (class in composer.core)
algorithms (composer.core.State attribute)
Alibi (class in composer.algorithms.alibi)
AlibiHparams (class in composer.algorithms.alibi)
apply() (composer.algorithms.augmix.AugMix method)
(composer.algorithms.blurpool.BlurPool method)
(composer.algorithms.channels_last.ChannelsLast method)
(composer.algorithms.cutout.CutOut method)
(composer.algorithms.label_smoothing.LabelSmoothing method)
(composer.algorithms.mixup.MixUp method)
(composer.algorithms.scale_schedule.ScaleSchedule method)
(composer.algorithms.stochastic_depth.StochasticDepth method)
(composer.core.Algorithm method)
apply_alibi() (in module composer.algorithms.alibi)
apply_blurpool() (in module composer.algorithms.blurpool)
(in module composer.algorithms.functional)
apply_ghost_batchnorm() (in module composer.algorithms.ghost_batchnorm)
apply_se() (in module composer.algorithms.functional)
(in module composer.algorithms.squeeze_excite)
apply_stochastic_depth (class in composer.algorithms.stochastic_depth)
apply_stochastic_depth() (in module composer.algorithms.stochastic_depth)
as_batch_dict() (in module composer.core.types)
as_batch_pair() (in module composer.core.types)
augment_and_mix() (in module composer.algorithms.augmix)
(in module composer.algorithms.functional)
AugMix (class in composer.algorithms.augmix)
,
[1]
AugMixHparams (class in composer.algorithms.augmix)
B
BaseLoggerBackend (class in composer.core.logging.base_backend)
BaseLoggerBackendHparams (class in composer.loggers.logger_hparams)
BaseMosaicModel (class in composer.models)
Batch (class in composer.core.types)
batch (composer.core.State attribute)
batch_dict (composer.core.State property)
batch_idx (composer.core.State property)
batch_pair (composer.core.State property)
BatchDict (class in composer.core.types)
BatchPair (class in composer.core.types)
Benchmarker (class in composer.callbacks.benchmarker)
blur_2d() (in module composer.algorithms.blurpool)
BlurConv2d (class in composer.algorithms.blurpool)
BlurMaxPool2d (class in composer.algorithms.blurpool)
BlurPool (class in composer.algorithms.blurpool)
,
[1]
BlurPool2d (class in composer.algorithms.blurpool)
BlurPoolHparams (class in composer.algorithms.blurpool)
BratsDatasetHparams (class in composer.datasets)
BreakEpochException
C
Callback (class in composer.core.callback)
callbacks (composer.core.State attribute)
ChannelsLast (class in composer.algorithms.channels_last)
,
[1]
ChannelsLastHparams (class in composer.algorithms.channels_last)
CIFAR10_ResNet56 (class in composer.models)
CIFAR10DatasetHparams (class in composer.datasets)
collate_fn (composer.datasets.DataloaderSpec attribute)
ColOut (class in composer.algorithms.colout)
colout() (in module composer.algorithms.colout)
(in module composer.algorithms.functional)
ColOutHparams (class in composer.algorithms.colout)
ComposedScheduler (class in composer.optim)
composer.algorithms.alibi
module
composer.algorithms.augmix
module
composer.algorithms.blurpool
module
composer.algorithms.channels_last
module
composer.algorithms.colout
module
composer.algorithms.cutout
module
composer.algorithms.ghost_batchnorm
module
composer.algorithms.label_smoothing
module
composer.algorithms.layer_freezing
module
composer.algorithms.mixup
module
composer.algorithms.progressive_resizing
module
composer.algorithms.randaugment
module
composer.algorithms.sam
module
composer.algorithms.scale_schedule
module
composer.algorithms.selective_backprop
module
composer.algorithms.squeeze_excite
module
composer.algorithms.stochastic_depth
module
composer.algorithms.swa
module
composer.core.logging.logger
module
composer.core.surgery
module
compute() (composer.models.loss.CrossEntropyLoss method)
(composer.models.loss.Dice method)
(composer.models.nlp_metrics.LanguageCrossEntropyLoss method)
(composer.models.nlp_metrics.Perplexity method)
ConstantLR (class in composer.optim)
count_module_instances() (in module composer.core.surgery)
create_from_hparams() (composer.trainer.Trainer class method)
CrossEntropyLoss (class in composer.models.loss)
CudaDataLoader (class in composer.trainer.devices.device_gpu)
CutOut (class in composer.algorithms.cutout)
,
[1]
cutout() (in module composer.algorithms.cutout)
(in module composer.algorithms.functional)
CutOutHparams (class in composer.algorithms.cutout)
D
DataLoader (class in composer.core.types)
dataloader_to_device() (composer.trainer.devices.device.Device method)
(composer.trainer.devices.device_cpu.DeviceCPU method)
(composer.trainer.devices.device_gpu.DeviceGPU method)
DataloaderHparams (class in composer.datasets)
DataloaderSpec (class in composer.datasets)
Dataset (class in composer.core.types)
dataset (composer.datasets.DataloaderSpec attribute)
DatasetHparams (class in composer.datasets)
ddp_backend (composer.trainer.devices.device.Device property)
(composer.trainer.devices.device_cpu.DeviceCPU property)
(composer.trainer.devices.device_gpu.DeviceGPU property)
DecoupledAdamW (class in composer.optim)
DecoupledSGDW (class in composer.optim)
Device (class in composer.trainer.devices.device)
DeviceCPU (class in composer.trainer.devices.device_cpu)
DeviceGPU (class in composer.trainer.devices.device_gpu)
Dice (class in composer.models.loss)
drop_last (composer.datasets.DataloaderSpec attribute)
E
EfficientNetB0 (class in composer.models)
Engine (class in composer.core)
engine (composer.trainer.Trainer attribute)
epoch (composer.core.State attribute)
eval() (composer.trainer.Trainer method)
eval_batch_size (composer.core.State attribute)
eval_dataloader (composer.core.State attribute)
Event (class in composer.core.event)
exit_code (composer.core.Trace attribute)
F
FileLoggerBackend (class in composer.loggers.file_logger)
FileLoggerBackendHparams (class in composer.loggers.logger_hparams)
find_unused_parameters (composer.core.Algorithm property)
fit() (composer.trainer.Trainer method)
forward() (composer.models.BaseMosaicModel method)
(composer.models.MosaicClassifier method)
(composer.models.MosaicTransformer method)
freeze_layers() (in module composer.algorithms.functional)
(in module composer.algorithms.layer_freezing)
G
gen_interpolation_lambda() (in module composer.algorithms.mixup.mixup)
generator (composer.datasets.DataloaderSpec attribute)
get_model_inputs() (composer.models.MosaicTransformer method)
GhostBatchNorm (class in composer.algorithms.ghost_batchnorm)
GhostBatchNormHparams (class in composer.algorithms.ghost_batchnorm)
grad_accum (composer.core.State attribute)
GradMonitor (class in composer.callbacks.grad_monitor)
I
ImagenetDatasetHparams (class in composer.datasets)
initialize_object() (composer.datasets.BratsDatasetHparams method)
(composer.datasets.CIFAR10DatasetHparams method)
(composer.datasets.DataloaderHparams method)
(composer.datasets.DatasetHparams method)
(composer.datasets.ImagenetDatasetHparams method)
(composer.datasets.LMDatasetHparams method)
(composer.datasets.MNISTDatasetHparams method)
(composer.datasets.SyntheticDatasetHparams method)
(composer.loggers.logger_hparams.BaseLoggerBackendHparams method)
(composer.loggers.logger_hparams.FileLoggerBackendHparams method)
(composer.loggers.logger_hparams.TQDMLoggerBackendHparams method)
(composer.loggers.logger_hparams.WandBLoggerBackendHparams method)
J
JSON (class in composer.core.types)
L
LabelSmoothing (class in composer.algorithms.label_smoothing)
,
[1]
LabelSmoothingHparams (class in composer.algorithms.label_smoothing)
LanguageCrossEntropyLoss (class in composer.models.nlp_metrics)
last_batch_size (composer.core.State attribute)
LayerFreezing (class in composer.algorithms.layer_freezing)
LayerFreezingHparams (class in composer.algorithms.layer_freezing)
LMDatasetHparams (class in composer.datasets)
load_state_dict() (composer.callbacks.speed_monitor.SpeedMonitor method)
(composer.callbacks.torch_profiler.TorchProfiler method)
(composer.core.State method)
(composer.core.types.Serializable method)
(composer.loggers.tqdm_logger.TQDMLoggerBackend method)
(composer.trainer.devices.device_cpu.DeviceCPU method)
(composer.trainer.devices.device_gpu.DeviceGPU method)
log_metric() (composer.core.logging.base_backend.BaseLoggerBackend method)
(composer.core.logging.base_backend.RankZeroLoggerBackend method)
Logger (class in composer.core.logging.logger)
logger (composer.trainer.Trainer attribute)
LogLevel (class in composer.core.logging.logger)
loss (composer.core.State attribute)
loss() (composer.models.BaseMosaicModel method)
(composer.models.MosaicClassifier method)
(composer.models.MosaicTransformer method)
LRMonitor (class in composer.callbacks.lr_monitor)
M
match() (composer.algorithms.augmix.AugMix method)
(composer.algorithms.blurpool.BlurPool method)
(composer.algorithms.channels_last.ChannelsLast method)
(composer.algorithms.cutout.CutOut method)
(composer.algorithms.label_smoothing.LabelSmoothing method)
(composer.algorithms.mixup.MixUp method)
(composer.algorithms.scale_schedule.ScaleSchedule method)
(composer.algorithms.stochastic_depth.StochasticDepth method)
(composer.core.Algorithm method)
max_epochs (composer.core.State attribute)
metric() (composer.core.logging.logger.Logger method)
metric_batch() (composer.core.logging.logger.Logger method)
metric_epoch() (composer.core.logging.logger.Logger method)
metric_fit() (composer.core.logging.logger.Logger method)
metric_microbatch() (composer.core.logging.logger.Logger method)
metric_verbose() (composer.core.logging.logger.Logger method)
Metrics (class in composer.core.types)
metrics() (composer.models.BaseMosaicModel method)
(composer.models.MosaicClassifier method)
(composer.models.MosaicTransformer method)
MixUp (class in composer.algorithms.mixup)
,
[1]
mixup_batch() (in module composer.algorithms.functional)
(in module composer.algorithms.mixup)
(in module composer.algorithms.mixup.mixup)
MixUpHparams (class in composer.algorithms.mixup)
MNIST_Classifier (class in composer.models)
MNISTDatasetHparams (class in composer.datasets)
Model (class in composer.core.types)
model (composer.core.State attribute)
ModelParameters (class in composer.core.types)
module
composer.algorithms.alibi
composer.algorithms.augmix
composer.algorithms.blurpool
composer.algorithms.channels_last
composer.algorithms.colout
composer.algorithms.cutout
composer.algorithms.ghost_batchnorm
composer.algorithms.label_smoothing
composer.algorithms.layer_freezing
composer.algorithms.mixup
composer.algorithms.progressive_resizing
composer.algorithms.randaugment
composer.algorithms.sam
composer.algorithms.scale_schedule
composer.algorithms.selective_backprop
composer.algorithms.squeeze_excite
composer.algorithms.stochastic_depth
composer.algorithms.swa
composer.core.logging.logger
composer.core.surgery
module_to_device() (composer.trainer.devices.device.Device method)
(composer.trainer.devices.device_cpu.DeviceCPU method)
(composer.trainer.devices.device_gpu.DeviceGPU method)
MosaicClassifier (class in composer.models)
MosaicTransformer (class in composer.models)
multiprocessing_context (composer.datasets.DataloaderSpec attribute)
N
nproc_per_node (composer.trainer.devices.device.Device property)
(composer.trainer.devices.device_cpu.DeviceCPU property)
(composer.trainer.devices.device_gpu.DeviceGPU property)
O
Optimizer (class in composer.core.types)
optimizer_to_device() (composer.trainer.devices.device.Device method)
Optimizers (class in composer.core.types)
optimizers (composer.core.State attribute)
order (composer.core.Trace attribute)
outputs (composer.core.State attribute)
P
Perplexity (class in composer.models.nlp_metrics)
Precision (class in composer.core.types)
precision (composer.core.State attribute)
precision_context() (composer.trainer.devices.device.Device method)
(composer.trainer.devices.device_cpu.DeviceCPU method)
(composer.trainer.devices.device_gpu.DeviceGPU method)
prefetch_fn (composer.datasets.DataloaderSpec attribute)
prepare() (composer.trainer.devices.device.Device method)
(composer.trainer.devices.device_cpu.DeviceCPU method)
(composer.trainer.devices.device_gpu.DeviceGPU method)
ProgressiveResizing (class in composer.algorithms.progressive_resizing)
ProgressiveResizingHparams (class in composer.algorithms.progressive_resizing)
R
RandAugment (class in composer.algorithms.randaugment)
randaugment() (in module composer.algorithms.functional)
(in module composer.algorithms.randaugment)
RandAugmentHparams (class in composer.algorithms.randaugment)
RankZeroCallback (class in composer.core.callback)
RankZeroLoggerBackend (class in composer.core.logging.base_backend)
replace_module_classes() (in module composer.core.surgery)
ReplacementFunction (class in composer.core.surgery)
resize_inputs() (in module composer.algorithms.functional)
(in module composer.algorithms.progressive_resizing)
ResNet101 (class in composer.models)
ResNet18 (class in composer.models)
ResNet50 (class in composer.models)
run (composer.core.Trace attribute)
run_event() (composer.core.Engine method)
S
SAM (class in composer.algorithms.sam)
SAMHparams (class in composer.algorithms.sam)
scale_scheduler() (in module composer.algorithms.functional)
(in module composer.algorithms.scale_schedule)
Scaler (class in composer.core.types)
scaler (composer.core.State attribute)
ScaleSchedule (class in composer.algorithms.scale_schedule)
,
[1]
ScaleScheduleHparams (class in composer.algorithms.scale_schedule)
Scheduler (class in composer.core.types)
Schedulers (class in composer.core.types)
schedulers (composer.core.State attribute)
selective_backprop() (in module composer.algorithms.functional)
SelectiveBackprop (class in composer.algorithms.selective_backprop)
SelectiveBackpropHparams (class in composer.algorithms.selective_backprop)
Serializable (class in composer.core.types)
shuffle (composer.datasets.DataloaderSpec attribute)
smooth_labels (class in composer.algorithms.label_smoothing)
smooth_labels() (in module composer.algorithms.functional)
(in module composer.algorithms.label_smoothing)
soft_cross_entropy() (in module composer.models.loss)
SpeedMonitor (class in composer.callbacks.speed_monitor)
split_fn (composer.datasets.DataloaderSpec attribute)
SqueezeExcite (class in composer.algorithms.squeeze_excite)
SqueezeExcite2d (class in composer.algorithms.squeeze_excite)
SqueezeExciteConv2d (class in composer.algorithms.squeeze_excite)
SqueezeExciteHparams (class in composer.algorithms.squeeze_excite)
State (class in composer.core)
state (composer.trainer.Trainer attribute)
state_dict() (composer.callbacks.speed_monitor.SpeedMonitor method)
(composer.callbacks.torch_profiler.TorchProfiler method)
(composer.core.State method)
(composer.core.types.Serializable method)
(composer.loggers.tqdm_logger.TQDMLoggerBackend method)
(composer.loggers.wandb_logger.WandBLoggerBackend method)
(composer.trainer.devices.device_cpu.DeviceCPU method)
(composer.trainer.devices.device_gpu.DeviceGPU method)
StateDict (class in composer.core.types)
step (composer.core.State attribute)
steps_per_epoch (composer.core.State property)
StochasticBottleneck (class in composer.algorithms.stochastic_depth)
StochasticDepth (class in composer.algorithms.stochastic_depth)
,
[1]
StochasticDepthHparams (class in composer.algorithms.stochastic_depth)
SWA (class in composer.algorithms.swa)
SWAHparams (class in composer.algorithms.swa)
SyntheticDatasetHparams (class in composer.datasets)
T
Tensor (class in composer.core.types)
tensor_to_device() (composer.trainer.devices.device.Device method)
(composer.trainer.devices.device_cpu.DeviceCPU method)
(composer.trainer.devices.device_gpu.DeviceGPU method)
Tensors (class in composer.core.types)
TorchProfiler (class in composer.callbacks.torch_profiler)
TPrefetchFn (class in composer.core.types)
TQDMLoggerBackend (class in composer.loggers.tqdm_logger)
TQDMLoggerBackendHparams (class in composer.loggers.logger_hparams)
Trace (class in composer.core)
train_batch_size (composer.core.State attribute)
train_dataloader (composer.core.State attribute)
Trainer (class in composer.trainer)
U
update() (composer.models.loss.CrossEntropyLoss method)
(composer.models.loss.Dice method)
(composer.models.nlp_metrics.LanguageCrossEntropyLoss method)
V
validate() (composer.models.BaseMosaicModel method)
(composer.models.MosaicClassifier method)
(composer.models.MosaicTransformer method)
W
WandBLoggerBackend (class in composer.loggers.wandb_logger)
WandBLoggerBackendHparams (class in composer.loggers.logger_hparams)
WarmUpLR (class in composer.optim)
will_log() (composer.core.logging.base_backend.BaseLoggerBackend method)
(composer.core.logging.base_backend.RankZeroLoggerBackend method)
worker_init_fn (composer.datasets.DataloaderSpec attribute)
Read the Docs
v: v0.2.3
Versions
latest
stable
v0.3.1
v0.3.0
v0.2.4
v0.2.3
Downloads
On Read the Docs
Project Home
Builds