composer.core.time#
Utilities to track training progress in terms of epochs, batches, samples, and tokens.
Callbacks, algorithms, and schedulers can use the current training time to fire at certain points in the training process.
The Timer
class tracks the total number of epochs, batches, samples, and tokens. The trainer is
responsible for updating it at the end of every epoch and batch. There is only one instance of the
Timer
, which is attached to the State
.
The Time
class represents static durations of training time or points in the training process in terms
of a specific TimeUnit
enum. This class supports comparisons, arithmetic, and conversions.
See the Time Guide for more details on tracking time during training.
Classes
Time represents static durations of training time or points in the training process in terms of a |
|
Enum class to represent units of time for the training process. |
|
Timer tracks the current training progress, in terms of epochs, batches, samples, and tokens. |
|
Timestamp represents a snapshot of |
- class composer.core.time.Time(value, unit)[source]#
Bases:
Generic
[composer.core.time.TValue
]Time represents static durations of training time or points in the training process in terms of a
TimeUnit
enum (epochs, batches, samples, tokens, or duration).See the Time Guide for more details on tracking time during training.
To construct an instance of
Time
, you can either:Use a value followed by a
TimeUnit
enum or string. For example,
>>> Time(5, TimeUnit.EPOCH) # describes 5 epochs. Time(5, TimeUnit.EPOCH) >>> Time(30_000, "tok") # describes 30,000 tokens. Time(30000, TimeUnit.TOKEN) >>> Time(0.5, "dur") # describes 50% of the training process. Time(0.5, TimeUnit.DURATION)
Use one of the helper methods. See:
Time
supports addition and subtraction with otherTime
instances that share the sameTimeUnit
. For example:>>> Time(1, TimeUnit.EPOCH) + Time(2, TimeUnit.EPOCH) Time(3, TimeUnit.EPOCH)
Time
supports multiplication. The multiplier must be either a number or have units ofTimeUnit.DURATION
. The multiplicand is scaled, and its units are kept.>>> Time(2, TimeUnit.EPOCH) * 0.5 Time(1, TimeUnit.EPOCH)
>>> Time(2, TimeUnit.EPOCH) * Time(0.5, TimeUnit.DURATION) Time(1, TimeUnit.EPOCH)
Time
supports division. If the divisor is an instance ofTime
, then it must have the same units as the dividend, and the result has units ofTimeUnit.DURATION
. For example:>>> Time(4, TimeUnit.EPOCH) / Time(2, TimeUnit.EPOCH) Time(2.0, TimeUnit.DURATION)
If the divisor is number, then the dividend is scaled, and it keeps its units. For example:
>>> Time(4, TimeUnit.EPOCH) / 2 Time(2, TimeUnit.EPOCH)
- Parameters
- classmethod from_batch(batch)[source]#
Create a
Time
with units ofTimeUnit.BATCH
. Equivalent toTime(batch, TimeUnit.BATCH)
.
- classmethod from_duration(duration)[source]#
Create a
Time
with units ofTimeUnit.DURATION
. Equivalent toTime(duration, TimeUnit.DURATION)
.
- classmethod from_epoch(epoch)[source]#
Create a
Time
with units ofTimeUnit.EPOCH
. Equivalent toTime(epoch, TimeUnit.EPOCH)
.
- classmethod from_sample(sample)[source]#
Create a
Time
with units ofTimeUnit.SAMPLE
. Equivalent toTime(sample, TimeUnit.SAMPLE)
.
- classmethod from_timestring(timestring)[source]#
Parse a time string into a
Time
instance. A time string is a numerical value followed by the value of aTimeUnit
enum. For example:>>> Time.from_timestring("5ep") # describes 5 epochs. Time(5, TimeUnit.EPOCH) >>> Time.from_timestring("3e4tok") # describes 30,000 tokens. Time(30000, TimeUnit.TOKEN) >>> Time.from_timestring("0.5dur") # describes 50% of the training process. Time(0.5, TimeUnit.DURATION)
- Returns
Time โ An instance of
Time
.
- classmethod from_token(token)[source]#
Create a
Time
with units ofTimeUnit.TOKEN
. Equivalent toTime(sample, TimeUnit.TOKEN)
.
- to_timestring()[source]#
Get the time-string representation.
For example:
>>> Time(5, TimeUnit.EPOCH).to_timestring() '5ep'
- Returns
str โ The time-string representation.
- property unit#
The unit of the time.
- property value#
The value of the time, as a number.
- class composer.core.time.TimeUnit(value)[source]#
Bases:
composer.utils.string_enum.StringEnum
Enum class to represent units of time for the training process.
- class composer.core.time.Timer[source]#
Bases:
composer.core.serializable.Serializable
Timer tracks the current training progress, in terms of epochs, batches, samples, and tokens.
See the Time Guide for more details on tracking time during training.
Note
An instance of this class is automatically constructed by the
State
constructor. A user need not instantiate this class.- property batch#
The total batch count.
- property batch_in_epoch#
The batch count in the current epoch (resets at 0 at the beginning of every epoch).
- property epoch#
The total epoch count.
- get_timestamp()[source]#
Returns a snapshot of the current time.
Unlike the
Timer
, the values in aTimestamp
are a snapshot and are NOT incremented as training progresses.- Returns
Timestamp โ A snapshot of the current training time.
- on_batch_complete(samples=0, tokens=0)[source]#
Called by the trainer at the end of every optimization batch.
Note
For accurate time tracking, the trainer is responsible for accumulating the total number of samples and/or tokens trained across all ranks before invoking this function.
- property sample#
The total sample count.
- property sample_in_epoch#
The sample count in the current epoch (resets at 0 at the beginning of every epoch).
- property token#
The total token count.
- property token_in_epoch#
The token count in the current epoch (resets at 0 at the beginning of every epoch).
- class composer.core.time.Timestamp(epoch, batch, batch_in_epoch, sample, sample_in_epoch, token, token_in_epoch)[source]#
Bases:
tuple
Timestamp represents a snapshot of
Timer
.It is returned from a call to
Timer.get_timestamp()
.Unlike the
Timer
, the values in aTimestamp
are a snapshot and are NOT incremented as training progresses.See the Time Guide for more details on tracking time during training.
Note
Timestamp
should not be instantiated directly; instead useTimer.get_timestamp()
.- batch_in_epoch#
The batch count in the epoch when the :class`Timestamp` was generated.
- sample_in_epoch#
The sample count in the epoch when the :class`Timestamp` was generated.