composer.models.gpt2.model#

composer.models.gpt2.model

Classes

ComposerTransformer

Implements the base logic that all Transformers can build on top of.

GPT2Model

Implements a GPT-2 wrapper around a ComposerTransformer.

MetricCollection

MetricCollection class can be used to chain metrics that have the same call pattern into one single class.

Perplexity

Subclasses LanguageCrossEntropyLoss to implement perplexity.

Attributes

  • Mapping

  • TYPE_CHECKING

  • annotations

class composer.models.gpt2.model.GPT2Model(module, config, tokenizer, gradient_checkpointing=False)[source]#

Bases: composer.models.transformer_shared.ComposerTransformer

Implements a GPT-2 wrapper around a ComposerTransformer.

See this paper for details on the GPT-2 architecutre.

Parameters

  • module (transformers.GPT2Model) – The model to wrap with this module.

  • config (transformers.GPT2Config) – The config for the model.

  • tokenizer (transformers.GPT2Tokenizer) – The tokenizer used for this model, necessary to assert required model inputs.