trainer_hparams#

The Hparams used to construct the Trainer.

Hparams

These classes are used with yahp for YAML-based configuration.

`EvalHparams`	Hyperparameters to describe a call to `Trainer.eval()`.
`ExperimentHparams`	Hyperparameters to describe an experiment.
`FitHparams`	Hyperparameters to describe a call to `Trainer.fit()`.
`TrainerHparams`	Params for instantiating the `Trainer`.

Methods

load()

class composer.trainer.trainer_hparams.EvalHparams(dataset, batch_size, dataloader_label='eval', subset_num_batches=- 1, log_level=LogLevel.FIT, metric_names=None)[source]#

Bases: yahp.hparams.Hparams

Hyperparameters to describe a call to Trainer.eval().

Parameters

dataset (DatasetHparams) – Dataset hyperparameters.
batch_size (int) – The evaluation batch size across all workers.
dataloader_label (str, optional) – See Trainer.eval().
subset_num_batches (int, optional) – See Trainer.eval().
log_level (LogLevel, optional) – See Trainer.eval().
metric_names (List[str], optional) –
Name of the metrics for the evaluator. (default: None)

Can be a torchmetrics metric name or the class name of a metric returned by ComposerModel.metrics(). If None (the default), uses all metrics in the model.

initialize_object(model, dataloader_hparams)[source]#

Construct a kwargs dictionary that can be unpacked and passed into Trainer.eval().

Parameters

model (ComposerModel) – The model.
dataloader_hparams (DataLoaderHparams) – The dataloader hyperparameters.

Returns

EvalKwargs – A kwargs dictionary that can be unpacked and passed into Trainer.eval().

class composer.trainer.trainer_hparams.ExperimentHparams(trainer, fits=<factory>, evals=<factory>)[source]#

Bases: yahp.hparams.Hparams

Hyperparameters to describe an experiment.

Unlike the TrainerHparams, this class allows for multiple configurations to Trainer.fit() and Trainer.eval() to also be specified.

Example usage:

experiment_hparams = ExperimentHparams(
    trainer=trainer_hparams,
    fits=[fit_1_hparams, fit_2_hparams],
    evals=[eval_1_hparams, eval_2_hparams],
)

trainer, fits, evals = experiment_hparams.initialize_object()

# The caller can invoke `trainer.fit(...)` and `trainer.eval(...)` in whatever
# order or combination makes sense for the experiment.
# In this example, all of the evaluations are run for each call to `trainer.fit(...)`.

for fit_kwargs in fits:
    trainer.fit(**fit_kwargs)
    for eval_kwargs in evals:
        trainer.eval(**eval_kwargs)

Parameters

trainer (TrainerHparams) – The trainer hparams.
fits (List[FitHparams]) – The hparams for calls to Trainer.fit().
evals (List[EvalHparams]) – The hparams for calls to Trainer.eval().

initialize_object()[source]#

Construct the Trainer, fit() kwargs, and eval() kwargs.

Returns: Tuple[Trainer, List[FitKwargs], List[EvalKwargs]] – A tuple of the (trainer, list of fit() kwargs, list of eval() kwargs).

class composer.trainer.trainer_hparams.FitHparams(train_dataset, train_batch_size=None, train_dataloader_label='train', train_subset_num_batches=None, compute_training_metrics=None, reset_time=False, duration=None, schedulers=None, scale_schedule_ratio=1.0, step_schedulers_every_batch=None, eval_dataset=None, evaluators=None, eval_batch_size=None, eval_interval=1, eval_subset_num_batches=- 1, precision=None, grad_accum=None)[source]#

Bases: yahp.hparams.Hparams

Hyperparameters to describe a call to Trainer.fit().

Parameters

train_dataset (DatasetHparams, optional) – Train dataset hyperparameters.
train_batch_size (int, optional) – The optimization batch size for the training dataset.
train_dataloader_label (str, optional) – See Trainer.fit().
train_subset_num_batches (int, optional) – See Trainer.fit().
compute_training_metrics (bool, optional) – See Trainer.fit().
reset_time (bool, optional) – See Trainer.fit().
duration (int | str, optional) – See Trainer.fit().
schedulers (List[SchedulerHparams], optional) – Scheduler hyperparameters.
scale_schedule_ratio (float, optional) – See Trainer.fit().
step_schedulers_every_batch (bool, optional) – See Trainer.fit().
eval_dataset (DatasetHparams, optional) – Validation dataset hyperameters. Cannot be specified with evaluators.
evaluators (EvaluatorHparams, optional) – Evaluator hyperparameters. Cannot be specified with eval_dataset.
eval_batch_size (int, optional) – See Trainer.fit().
eval_interval (int | str, optional) – See Trainer.fit().
eval_subset_num_batches (int, optional) – See Trainer.fit().
precision (Precision, optional) – See Trainer.fit().
grad_accum (int, optional) – See Trainer.fit().

initialize_object(model, dataloader_hparams)[source]#

Construct a kwargs dictionary that can be unpacked and passed into Trainer.fit().

Parameters

model (ComposerModel) – The model.
dataloader_hparams (DataLoaderHparams) – The dataloader hyperparameters.

Returns

FitKwargs – A kwargs dictionary that can be unpacked and passed into Trainer.fit().

class composer.trainer.trainer_hparams.TrainerHparams(model, dataloader=DataLoaderHparams(num_workers=8, prefetch_factor=2, persistent_workers=True, pin_memory=True, timeout=0.0), train_dataset=None, train_dataloader_label='train', train_batch_size=None, train_subset_num_batches=- 1, compute_training_metrics=False, max_duration=None, algorithms=None, optimizers=None, schedulers=None, scale_schedule_ratio=1.0, step_schedulers_every_batch=None, val_dataset=None, evaluators=None, eval_batch_size=None, eval_interval=1, eval_subset_num_batches=- 1, callbacks=None, loggers=None, run_name=None, progress_bar=True, log_to_console=None, console_log_level=LogLevel.EPOCH, console_stream='stderr', python_log_level='INFO', load_path=None, load_object_store=None, load_logger_destination=None, load_weights_only=False, load_strict_model_weights=False, load_ignore_keys=None, load_progress_bar=True, save_folder=None, save_filename='ep{epoch}-ba{batch}-rank{rank}.pt', save_artifact_name='{run_name}/checkpoints/ep{epoch}-ba{batch}-rank{rank}', save_latest_filename='latest-rank{rank}.pt', save_latest_artifact_name='{run_name}/checkpoints/latest-rank{rank}', save_overwrite=False, save_weights_only=False, save_interval='1ep', save_num_checkpoints_to_keep=- 1, autoresume=False, deepspeed_config=None, device=None, precision=None, grad_accum=1, seed=None, deterministic_mode=False, dist_timeout=300.0, ddp_sync_strategy=None, grad_clip_norm=- 1.0, profiler=None)[source]#

Bases: yahp.hparams.Hparams

Params for instantiating the Trainer.