composer.trainer.trainer_hparams#
The Hparams
used to construct the Trainer
.
Hparams
These classes are used with yahp
for YAML
-based configuration.
|
Hyperparameters to describe a call to |
|
Hyperparameters to describe an experiment. |
|
Hyperparameters to describe a call to |
|
Params for instantiating the |
Methods
load()
- class composer.trainer.trainer_hparams.EvalHparams(dataset, batch_size, dataloader_label='eval', subset_num_batches=- 1, log_level=LogLevel.FIT, metric_names=None)[source]#
Bases:
yahp.hparams.Hparams
Hyperparameters to describe a call to
Trainer.eval()
.- Parameters
dataset (DatasetHparams) โ Dataset hyperparameters.
batch_size (int) โ The evaluation batch size across all workers.
dataloader_label (str, optional) โ See
Trainer.eval()
.subset_num_batches (int, optional) โ See
Trainer.eval()
.log_level (LogLevel, optional) โ See
Trainer.eval()
.metric_names (List[str], optional) โ
Name of the metrics for the evaluator. (default:
None
)Can be a
torchmetrics
metric name or the class name of a metric returned byComposerModel.metrics()
. If None (the default), uses all metrics in the model.
- initialize_object(model, dataloader_hparams)[source]#
Construct a kwargs dictionary that can be unpacked and passed into
Trainer.eval()
.- Parameters
model (ComposerModel) โ The model.
dataloader_hparams (DataLoaderHparams) โ The dataloader hyperparameters.
- Returns
EvalKwargs โ A kwargs dictionary that can be unpacked and passed into
Trainer.eval()
.
- class composer.trainer.trainer_hparams.ExperimentHparams(trainer, fits=<factory>, evals=<factory>)[source]#
Bases:
yahp.hparams.Hparams
Hyperparameters to describe an experiment.
Unlike the
TrainerHparams
, this class allows for multiple configurations toTrainer.fit()
andTrainer.eval()
to also be specified.Example usage:
experiment_hparams = ExperimentHparams( trainer=trainer_hparams, fits=[fit_1_hparams, fit_2_hparams], evals=[eval_1_hparams, eval_2_hparams], ) trainer, fits, evals = experiment_hparams.initialize_object() # The caller can invoke `trainer.fit(...)` and `trainer.eval(...)` in whatever # order or combination makes sense for the experiment. # In this example, all of the evaluations are run for each call to `trainer.fit(...)`. for fit_kwargs in fits: trainer.fit(**fit_kwargs) for eval_kwargs in evals: trainer.eval(**eval_kwargs)
- Parameters
trainer (TrainerHparams) โ The trainer hparams.
fits (List[FitHparams]) โ The hparams for calls to
Trainer.fit()
.evals (List[EvalHparams]) โ The hparams for calls to
Trainer.eval()
.
- class composer.trainer.trainer_hparams.FitHparams(train_dataset, train_batch_size=None, train_dataloader_label='train', train_subset_num_batches=None, compute_training_metrics=None, reset_time=False, duration=None, schedulers=None, scale_schedule_ratio=1.0, step_schedulers_every_batch=None, eval_dataset=None, evaluators=None, eval_batch_size=None, eval_interval=1, eval_subset_num_batches=- 1, precision=None, grad_accum=None)[source]#
Bases:
yahp.hparams.Hparams
Hyperparameters to describe a call to
Trainer.fit()
.- Parameters
train_dataset (DatasetHparams, optional) โ Train dataset hyperparameters.
train_batch_size (int, optional) โ The optimization batch size for the training dataset.
train_dataloader_label (str, optional) โ See
Trainer.fit()
.train_subset_num_batches (int, optional) โ See
Trainer.fit()
.compute_training_metrics (bool, optional) โ See
Trainer.fit()
.reset_time (bool, optional) โ See
Trainer.fit()
.duration (int | str, optional) โ See
Trainer.fit()
.schedulers (List[SchedulerHparams], optional) โ Scheduler hyperparameters.
scale_schedule_ratio (float, optional) โ See
Trainer.fit()
.step_schedulers_every_batch (bool, optional) โ See
Trainer.fit()
.eval_dataset (DatasetHparams, optional) โ Validation dataset hyperameters. Cannot be specified with
evaluators
.evaluators (EvaluatorHparams, optional) โ Evaluator hyperparameters. Cannot be specified with
eval_dataset
.eval_batch_size (int, optional) โ See
Trainer.fit()
.eval_interval (int | str, optional) โ See
Trainer.fit()
.eval_subset_num_batches (int, optional) โ See
Trainer.fit()
.precision (Precision, optional) โ See
Trainer.fit()
.grad_accum (int, optional) โ See
Trainer.fit()
.
- initialize_object(model, dataloader_hparams)[source]#
Construct a kwargs dictionary that can be unpacked and passed into
Trainer.fit()
.- Parameters
model (ComposerModel) โ The model.
dataloader_hparams (DataLoaderHparams) โ The dataloader hyperparameters.
- Returns
FitKwargs โ A kwargs dictionary that can be unpacked and passed into
Trainer.fit()
.
- class composer.trainer.trainer_hparams.TrainerHparams(model, dataloader=DataLoaderHparams(num_workers=8, prefetch_factor=2, persistent_workers=True, pin_memory=True, timeout=0.0), train_dataset=None, train_dataloader_label='train', train_batch_size=None, train_subset_num_batches=- 1, compute_training_metrics=False, max_duration=None, algorithms=None, optimizers=None, schedulers=None, scale_schedule_ratio=1.0, step_schedulers_every_batch=None, val_dataset=None, evaluators=None, eval_batch_size=None, eval_interval=1, eval_subset_num_batches=- 1, callbacks=None, loggers=None, run_name=None, progress_bar=True, log_to_console=None, console_log_level=LogLevel.EPOCH, console_stream='stderr', python_log_level='INFO', load_path=None, load_object_store=None, load_logger_destination=None, load_weights_only=False, load_strict_model_weights=False, load_ignore_keys=None, load_progress_bar=True, save_folder=None, save_filename='ep{epoch}-ba{batch}-rank{rank}', save_artifact_name='{run_name}/checkpoints/ep{epoch}-ba{batch}-rank{rank}', save_latest_filename='latest-rank{rank}', save_latest_artifact_name='{run_name}/checkpoints/latest-rank{rank}', save_overwrite=False, save_weights_only=False, save_interval='1ep', save_num_checkpoints_to_keep=- 1, autoresume=False, deepspeed_config=None, device=None, precision=None, grad_accum=1, seed=None, deterministic_mode=False, dist_timeout=300.0, ddp_sync_strategy=None, grad_clip_norm=- 1.0, profiler=None)[source]#
Bases:
yahp.hparams.Hparams
Params for instantiating the
Trainer
.See also
The documentation for the
Trainer
.- Parameters
model (ModelHparams) โ
Hparams for constructing the model to train.
See also
composer.models
for models built into Composer.dataloader (DataLoaderHparams) โ Hparams used for constructing the dataloader which will be used for loading the train dataset and (if provided) the validation dataset.
train_dataset (DatasetHparams) โ
Hparams used to construct the dataset used for training.
See also
composer.datasets
for datasets built into Composer.train_batch_size (int) โ The optimization batch size to use for training. This is the total batch size that is used to produce a gradient for the optimizer update step.
max_duration (str) โ
The maximum duration to train as a str (e.g.
1ep
, or10ba
). Will be converted to aTime
object.See also
Time
for more details on time construction.algorithms (List[AlgorithmHparams], optional) โ
The algorithms to use during training. (default:
[]
)See also
composer.algorithms
for the different algorithms built into Composer.optimizers (OptimizerHparams, optional) โ
The hparams for constructing the optimizer. (default:
None
)See also
Trainer
for the default optimizer behavior whenNone
is provided.See also
composer.optim
for the different optimizers built into Composer.schedulers (List[SchedulerHparams], optional) โ
The learning rate schedulers. (default:
[]
).See also
Trainer
for the default scheduler behavior when[]
is provided.See also
composer.optim.scheduler
for the different schedulers built into Composer.step_schedulers_every_batch (bool, optional) โ See
Trainer
.val_dataset (DatasetHparams, optional) โ
Hparams for constructing the dataset used for evaluation. (default:
None
)See also
composer.datasets
for datasets built into Composer.evaluators (List[EvaluatorHparams], optional) โ
Hparams for constructing evaluators to be used during the eval loop. Evaluators should be used when evaluating one or more specific metrics across one or more datasets. (default:
None
)See also
Evaluator
for more details on evaluators.eval_batch_size (int, optional) โ The batch size to use for evaluation. Must be provided if one of
val_dataset
orevaluators
is set. (default:None
)callbacks (List[CallbackHparams], optional) โ
Hparams to construct the callbacks to run during training. (default:
[]
)See also
composer.callbacks
for the different callbacks built into Composer.loggers (List[LoggerDestinationHparams], optional) โ
Hparams for constructing the destinations to log to. (default:
[]
)See also
composer.loggers
for the different loggers built into Composer.python_log_level (str) โ
The Python log level to use for log statements in the
composer
module. (default:INFO
)See also
The
logging
module in Python.load_object_store (ObjectStoreHparams, optional) โ See
Trainer
. Bothload_logger_destination
andload_object_store
should not be provided since there can only be one location to load from.load_logger_destination (LoggerDestination, optional) โ Used to specify a
LoggerDestination
forload_object_store
inTrainer
as Hparams doesnโt support a Union type for those objects. Bothload_logger_destination
andload_object_store
should not be provided since there can only be one location to load from.load_ignore_keys (List[str] | (Dict) -> None, optional) โ See
Trainer
.save_folder (str, optional) โ See
CheckpointSaver
.save_filename (str, optional) โ See
CheckpointSaver
.save_artifact_name (str, optional) โ See
CheckpointSaver
.save_latest_filename (str, optional) โ See
CheckpointSaver
.save_latest_artifact_name (str, optional) โ See
CheckpointSaver
.save_overwrite (str, optional) โ See
CheckpointSaver
.save_weights_only (bool, optional) โ See
CheckpointSaver
.save_interval (str, optional) โ See
CheckpointSaverHparams
.save_num_checkpoints_to_keep (int, optional) โ See
CheckpointSaver
.deepspeed_config (Dict[str, JSON], optional) โ If set to a dict will be used for as the DeepSpeed config for training (see
Trainer
for more details). IfNone
(the default), DeepSpeed will not be used.device (Device, optional) โ Hparams for constructing the device used for training. (default:
None
)ddp_sync_strategy (DDPSyncStrategy, optional) โ See
Trainer
.profiler (ProfilerHparams, optional) โ Profiler hyperparameters.