Timestamp

class composer.Timestamp(epoch=0, batch=0, sample=0, token=0, batch_in_epoch=0, sample_in_epoch=0, token_in_epoch=0, total_wct=None, epoch_wct=None, batch_wct=None)

Timestamp represents a snapshot of the current training progress.

The timestamp measures training progress in terms of epochs, batches, samples, tokens, and wall clock time. Timestamps are not updated in-place.

See the Time Guide for more details on tracking time during training.

Parameters

• epoch (int | Time[int], optional) – The epoch.

• batch (int | Time[int], optional) – the batch.

• sample (int | Time[int], optional) – The sample.

• token (int | Time[int], optional) – The token.

• batch_in_epoch (int | Time[int], optional) – The batch in the epoch.

• sample_in_epoch (int | Time[int], optional) – The sample in the epoch.

• token_in_epoch (int | Time[int], optional) – The token in the epoch.

• total_wct (timedelta, optional) – The total wall-clock duration.

• epoch_wct (timedelta, optional) – The wall-clock duration of the last epoch.

• batch_wct (timedelta, optional) – The wall-clock duration of the last batch.

property batch

The total batch count.

property batch_in_epoch

The batch count in the current epoch (resets at 0 at the beginning of every epoch).

property batch_wct

The wall-clock duration (in seconds) for the last batch.

copy(epoch=None, batch=None, sample=None, token=None, batch_in_epoch=None, sample_in_epoch=None, token_in_epoch=None, total_wct=None, epoch_wct=None, batch_wct=None)

Create a copy of the timestamp.

Any specified values will override the existing values in the returned copy.

Parameters

• epoch (int | Time[int], optional) – The epoch.

• batch (int | Time[int], optional) – the batch.

• sample (int | Time[int], optional) – The sample.

• token (int | Time[int], optional) – The token.

• batch_in_epoch (int | Time[int], optional) – The batch in the epoch.

• sample_in_epoch (int | Time[int], optional) – The sample in the epoch.

• token_in_epoch (int | Time[int], optional) – The token in the epoch.

• total_wct (timedelta, optional) – The elapsed duration from the beginning of training.

Returns

Timestamp – A new timestamp instance, created from a copy, but with any specified values overriding the existing values.

property epoch

The total epoch count.

property epoch_wct

The wall-clock duration (in seconds) for the current epoch.

get(unit)

Returns the current time in the specified unit.

Parameters

unit (str | TimeUnit) – The desired unit.

Returns

Time – The current time, in the specified unit.

get_state()

Returns all values of the timestamp object in a dictionary.

Returns

Dict[str, Union[Time[int], datetime.timedelta]] – All values of the timestamp object.

property sample

The total sample count.

property sample_in_epoch

The sample count in the current epoch (resets at 0 at the beginning of every epoch).

to_next_batch(samples=0, tokens=0, duration=None)

Create a new Timestamp, advanced to the next batch.

Equivalent to:

>>> timestamp.copy(
...     batch=timestamp.batch + 1,
...     batch_in_epoch=timestamp.batch_in_epoch + 1,
...     sample=timestamp.sample + samples,
...     sample_in_epoch=timestamp.sample_in_epoch + samples,
...     token = timestamp.token + tokens,
...     token_in_epoch=timestamp.token_in_epoch + tokens,
...     total_wct=timestamp.total_wct + duration,
...     epoch_wct=timestamp.epoch_wct + duration,
...     batch_wct=duration,
... )
Timestamp(...)

Note

For accurate time tracking, when doing distributed training, the samples and tokens should be the total across all ranks for the given batch. This method will not accumulate these counts automatically. If per-rank sample and token counts are provided, these counts will differ across ranks, which could lead towards inconsistent behavior by Algorithm or Callback instances that use these counts.

Parameters

• samples (int | Time, optional) – The number of samples trained in the batch. Defaults to 0.

• tokens (int | Time, optional) – The number of tokens trained in the batch. Defaults to 0.

• duration (timedelta, optional) – The duration to train the batch.

to_next_epoch()

Create a new Timestamp, advanced to the next epoch.

Equivalent to:

>>> timestamp.copy(
...     epoch=timestamp.epoch+1,
...     batch_in_epoch=0,
...     sample_in_epoch=0,
...     token_in_epoch=0,
...     epoch_wct=datetime.timedelta(seconds=0),
...     batch_wct=datetime.timedelta(seconds=0),
... )
Timestamp(...)

property token

The total token count.

property token_in_epoch

The token count in the current epoch (resets at 0 at the beginning of every epoch).

property total_wct

The wall-clock duration (in seconds) from the beginning of training.