composer.models.bert.model#

composer.models.bert.model

Classes

BERTModel

Implements a BERT wrapper around a ComposerTransformer.
class composer.models.bert.model.BERTModel(module, config, tokenizer)[source]#

Bases: composer.models.transformer_shared.ComposerTransformer

Implements a BERT wrapper around a ComposerTransformer.

See this paper for details on the BERT architecutre.

Parameters

  • module (transformers.BertModel) – The model to wrap with this module.

  • config (transformers.BertConfig) – The config for the model.

  • tokenizer (transformers.BertTokenizer) – The tokenizer used for this model, necessary to assert required model inputs.

validate(batch)[source]#

Runs the validation step.

Parameters

batch (BatchDict) – a dictionary of Dict[str, Tensor] of inputs that the model expects, as found in ComposerTransformer.get_model_inputs().

Returns

  • A tuple of (Tensor, Tensor)

  • This is fed into directly into the output of :meth:`metrics`.