composer.models.transformer_hparams#
YAHP hp.Hparams
hyperparameters for ComposerTransformers.
Hparams
These classes are used with yahp
for YAML
-based configuration.
Defines the necessary hyperparameters for a Transformer base module. |
- class composer.models.transformer_hparams.TransformerHparams(initializers=<factory>, num_classes=None, tokenizer_name=None, pretrained_model_name=None, model_config=<factory>, use_pretrained=False, gradient_checkpointing=False)[source]#
Bases:
composer.models.model_hparams.ModelHparams
,abc.ABC
Defines the necessary hyperparameters for a Transformer base module.
- Parameters
pretrained_model_name (Optional[str]) โ โPretrained model name to pull from Huggingface Model Hub.โ
model_config (Dict[str, JSON]) โ A dictionary providing a HuggingFace model configuration.
tokenizer_name (str) โ The tokenizer used for this model, necessary to assert required model inputs.
use_pretrained (bool, optional) โ Whether to initialize the model with the pretrained weights. Default:
False
gradient_checkpointing (bool, optional) โ Use gradient checkpointing. Default:
False
.