composer.optim.optimizer_hparams

Hyperparameters for optimizers.

Hparams

These classes are used with yahp for YAML-based configuration.

AdamHparams	Hyperparameters for the Adam optimizer.
AdamWHparams	Hyperparameters for the AdamW optimizer.
DecoupledAdamWHparams	Hyperparameters for the DecoupledAdamW optimizer.
DecoupledSGDWHparams	Hyperparameters for the DecoupledSGDW optimizer.
OptimizerHparams	Base class for optimizer hyperparameter classes.
RAdamHparams	Hyperparameters for the RAdam optimizer.
RMSpropHparams	Hyperparameters for the RMSprop optimizer.
SGDHparams	Hyperparameters for the SGD optimizer.

class composer.optim.optimizer_hparams.AdamHparams(lr=0.001, betas=<factory>, eps=1e-08, weight_decay=0.0, amsgrad=False)

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the Adam optimizer.

See Adam for documentation.

Parameters

• lr (float, optional) – See Adam.

• betas (float, optional) – See Adam.

• eps (float, optional) – See Adam.

• weight_decay (float, optional) – See Adam.

• amsgrad (bool, optional) – See Adam.

class composer.optim.optimizer_hparams.AdamWHparams(lr=0.001, betas=<factory>, eps=1e-08, weight_decay=0.01, amsgrad=False)

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the AdamW optimizer.

See AdamW for documentation.

Parameters

• lr (float, optional) – See AdamW.

• betas (float, optional) – See AdamW.

• eps (float, optional) – See AdamW.

• weight_decay (float, optional) – See AdamW.

• amsgrad (bool, optional) – See AdamW.

class composer.optim.optimizer_hparams.DecoupledAdamWHparams(lr=0.001, betas=<factory>, eps=1e-08, weight_decay=0.01, amsgrad=False)

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the DecoupledAdamW optimizer.

See DecoupledAdamW for documentation.

Parameters

• lr (float, optional) – See DecoupledAdamW.

• betas (float, optional) – See DecoupledAdamW.

• eps (float, optional) – See DecoupledAdamW.

• weight_decay (float, optional) – See DecoupledAdamW.

• amsgrad (bool, optional) – See DecoupledAdamW.

class composer.optim.optimizer_hparams.DecoupledSGDWHparams(lr, momentum=0.0, weight_decay=0.0, dampening=0.0, nesterov=False)

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the DecoupledSGDW optimizer.

See DecoupledSGDW for documentation.

Parameters

• lr (float) – See DecoupledSGDW.

• momentum (float, optional) – See DecoupledSGDW.

• weight_decay (float, optional) – See DecoupledSGDW.

• dampening (float, optional) – See DecoupledSGDW.

• nesterov (bool, optional) – See DecoupledSGDW.

class composer.optim.optimizer_hparams.OptimizerHparams

Bases: yahp.hparams.Hparams, abc.ABC

Base class for optimizer hyperparameter classes.

Optimizer parameters that are added to TrainerHparams (e.g. via YAML or the CLI) are initialized in the training loop.

initialize_object(param_group)

Initializes the optimizer.

Parameters

param_group (Iterable[Tensor] | Iterable[Dict[str, Tensor]]) – Parameters for this optimizer to optimize.

class composer.optim.optimizer_hparams.RAdamHparams(lr=0.001, betas=<factory>, eps=1e-08, weight_decay=0.0)

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the RAdam optimizer.

See RAdam for documentation.

Parameters

• lr (float, optional) – See RAdam.

• betas (float, optional) – See RAdam.

• eps (float, optional) – See RAdam.

• weight_decay (float, optional) – See RAdam.

class composer.optim.optimizer_hparams.RMSpropHparams(lr, alpha=0.99, eps=1e-08, momentum=0.0, weight_decay=0.0, centered=False)

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the RMSprop optimizer.

See RMSprop for documentation.

Parameters

• lr (float) – See RMSprop.

• alpha (float, optional) – See RMSprop.

• eps (float, optional) – See RMSprop.

• momentum (float, optional) – See RMSprop.

• weight_decay (float, optional) – See RMSprop.

• centered (bool, optional) – See RMSprop.

class composer.optim.optimizer_hparams.SGDHparams(lr, momentum=0.0, weight_decay=0.0, dampening=0.0, nesterov=False)

Bases: composer.optim.optimizer_hparams.OptimizerHparams

Hyperparameters for the SGD optimizer.

See SGD for documentation.

Parameters

• lr (float) – See SGD.

• momentum (float, optional) – See SGD.

• weight_decay (float, optional) – See SGD.

• dampening (float, optional) – See SGD.

• nesterov (bool, optional) – See SGD.