PolynomialScheduler#

class composer.optim.PolynomialScheduler(power, t_max='1dur', alpha_f=0.0)[source]#

Sets the learning rate to be proportional to a power of the fraction of training time left.

Specifially, the learning rate multiplier \(\alpha\) can be expressed as:

\[\alpha(t) = \alpha_f + (1 - \alpha_f) \times (1 - \tau) ^ {\kappa} \]

Given \(\tau\), the fraction of time elapsed (clipped to the interval \([0, 1]\)), as:

\[\tau = t / t_{max} \]

Where \(\kappa\) represents the exponent to be used for the proportionality relationship, \(t_{max}\) represents the duration of this scheduler, and \(\alpha_f\) represents the learning rate multiplier to decay to.

Parameters
  • power (float) โ€“ The exponent to be used for the proportionality relationship.

  • t_max (str | Time) โ€“ The duration of this scheduler. Default = "1dur".

  • alpha_f (float) โ€“ Learning rate multiplier to decay to. Default = 0.0.