Composer#

Composer is a library for training neural networks better, faster, and cheaper. It contains many state-of-the-art methods for accelerating neural network training and improving generalization, along with an optional Trainer API that makes composing many different enhancements easy.

We implemented dozens of papers and trained thousands of models so you donโ€™t have to, and built strong baselines to rigorously measure their effect.

With Composer, you can build recipes to:

  • Train ResNet-50 on ImageNet to the standard 76.6% top-one accuracy for $40 in 1.2 hours (with vanilla PyTorch: $116 in 3.5 hours) on AWS.

  • Train a GPT-2 125M to a standard perplexity of 24.11 for $145 in 4.5 hours (with vanilla PyTorch: $255 in 7.8 hours) on AWS.

Composer features:

  • 20+ curated efficiency methods for training better language and vision models.

  • An easy-to-use Trainer interface written to be as performant as possible, with integrated multi-GPU training, mixed-precision training, and numerous best practices implemented out of the box.

  • Standalone functional forms of training methods for easy integration into your own training loop

  • Strong, reproducible baselines to get you started as fast as possible.

See ๐Ÿ’พ Installation for how to install and start using Composer, โš™๏ธ Using the Trainer for an introduction to our trainer, and ๐Ÿค– Algorithms for details about our efficiency methods and how to use them in your code.

At MosaicML, we are focused on making fast, high-quality ML training accessible to everyone. To do this, we continually productionize state-of-the-art academic research on efficient model training, and also study the combinations of these methods in order to ensure that model training is โœจ as efficient as possible โœจ.

If you have any questions, please feel free to reach out to us on Twitter, Email, or Slack!

Composer is part of the broader Machine Learning community, and we welcome any contributions, pull requests, and issues.