composer.models.gpt2.gpt2_hparams#

composer.models.gpt2.gpt2_hparams

Hparams

These classes are used with yahp for YAML-based configuration.

GPT2Hparams

Overrides TransformerHparams to create GPT-2 specific models and configs.

TransformerHparams

Defines the necessary hyparameters for a Transformer base module.

Attributes

  • TYPE_CHECKING

class composer.models.gpt2.gpt2_hparams.GPT2Hparams(initializers=<factory>, num_classes=None, tokenizer_name=None, pretrained_model_name=None, model_config=<factory>, use_pretrained=False, gradient_checkpointing=False)[source]#

Bases: composer.models.transformer_hparams.TransformerHparams

Overrides TransformerHparams to create GPT-2 specific models and configs.