๐พ Installation#
Composer
is available with pip:
pip install mosaicml
as well as with Anaconda:
conda install -c mosaicml mosaicml
To include non-core dependencies that are required by some algorithms, callbacks, datasets, or models, the following installation targets are available:
pip install 'mosaicml[dev]'
: Installs development dependencies, which are required for running tests and building documentation.pip install 'mosaicml[deepspeed]'
: Installs Composer with support fordeepspeed
.pip install 'mosaicml[nlp]'
: Installs Composer with support for NLP models and algorithms.pip install 'mosaicml[unet]'
: Installs Composer with support for Unet.pip install 'mosaicml[timm]'
: Installs Composer with support fortimm
.pip install 'mosaicml[wandb]'
: Installs Composer with support forwandb
.pip install 'mosaicml[all]'
: Install all optional dependencies.
For a developer install, clone directly:
git clone https://github.com/mosaicml/composer.git
cd composer
pip install -e ".[all]"
Note
For fast loading of image data, we highly recommend installing Pillow-SIMD. To install, vanilla pillow must first be uninstalled.
pip uninstall pillow && pip install pillow-simd
Pillow-SIMD is not supported for Apple M1 Macs.
Docker#
To simplify the environment setup for Composer, we provide a set of convenient Docker Images.
The Base
flavor of Docker Images contains PyTorch pre-installed; the Vision
flavor also includes OpenCV,
MM Segmentation, and FFCV dependencies.
Linux Distro |
Flavor |
PyTorch Version |
CUDA Version |
Python Version |
Docker Tags |
---|---|---|---|---|---|
Ubuntu 20.04 |
Base |
1.11.0 |
11.3.1 |
3.9 |
|
Ubuntu 20.04 |
Base |
1.11.0 |
cpu |
3.9 |
|
Ubuntu 20.04 |
Base |
1.10.2 |
11.3.1 |
3.8 |
|
Ubuntu 20.04 |
Base |
1.10.2 |
cpu |
3.8 |
|
Ubuntu 20.04 |
Base |
1.9.1 |
11.1.1 |
3.7 |
|
Ubuntu 20.04 |
Base |
1.9.1 |
cpu |
3.7 |
|
Ubuntu 20.04 |
Vision |
1.11.0 |
11.3.1 |
3.9 |
|
Ubuntu 20.04 |
Vision |
1.11.0 |
cpu |
3.9 |
|
Pillow-SIMD
is installed by default in all images.
Note
These images do not come with Composer preinstalled. To install Composer, run pip install mosaicml
once inside the image.
Pre-built images can be pulled from MosaicMLโs DockerHub Repository:
docker pull mosaicml/pytorch
Building images locally#
# Build the default image
make
# Build composer with Python 3.8
PYTHON_VERSION=3.8 make
Note
Docker must be installed on your local machine.
๐ Quick Start#
Access our library of speedup methods with the ฦ() Functional API:
import logging
from composer import functional as CF
import torchvision.models as models
logging.basicConfig(level=logging.INFO)
model = models.resnet50()
CF.apply_blurpool(model)
This creates a ResNet50 model and replaces several pooling and convolution layers with BlurPool variants (Zhang et al, 2019). For more information, see ๐ BlurPool. The method should log:
Applied BlurPool to model ResNet. Model now has 1 BlurMaxPool2d and 6 BlurConv2D layers.
These methods are easy to integrate into your own training loop code with just a few lines.
For an overview of the algorithms, see ๐ค Algorithms.
We make composing recipes together even easier with our (optional) :class`.Trainer`. Here, we train an MNIST classifer with a recipe of methods:
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
from composer import Trainer
from composer.models import MNIST_Classifier
from composer.algorithms import LabelSmoothing, CutMix, ChannelsLast
transform = transforms.Compose([transforms.ToTensor()])
dataset = datasets.MNIST("data", train=True, download=True, transform=transform)
train_dataloader = DataLoader(dataset, batch_size=128)
trainer = Trainer(
model=MNIST_Classifier(num_classes=10),
train_dataloader=train_dataloader,
max_duration="2ep",
algorithms=[
LabelSmoothing(smoothing=0.1),
CutMix(num_classes=10),
ChannelsLast(),
]
)
trainer.fit()
We handle inserting and running the logic during the training so that any algorithms you specify โjust work.โ
Besides easily running our built-in algorithms, Composer also features:
An interface to flexibly add algorithms to the training loop
An engine that manages the ordering of algorithms for composition
A trainer to handle boilerplate around numerics, distributed training, and others
Integration with popular model libraries such as TIMM and HuggingFace Transformers
Next steps#
Try ๐ Notebooks to see some of our speed-ups with notebooks on Colab.
See โ๏ธ Using the Trainer for more details on our trainer.
Read ๐ Welcome Tour for a tour through the library.