composer.loggers.file_logger

Logs to a file.

Classes

FileLogger

Log data to a file.

class composer.loggers.file_logger.FileLogger(filename='{run_name}/logs-rank{rank}.txt', artifact_name=None, *, capture_stdout=True, capture_stderr=True, buffer_size=1, log_level=LogLevel.EPOCH, log_interval=1, flush_interval=100, overwrite=False)

Bases: composer.loggers.logger_destination.LoggerDestination

Log data to a file.

Example usage:

from composer.loggers import FileLogger, LogLevel
from composer.trainer import Trainer
file_logger = FileLogger(
    filename="{run_name}/logs-rank{rank}.txt",
    buffer_size=1,
    log_level=LogLevel.BATCH,
    log_interval=2,
    flush_interval=50
)
trainer = Trainer(
    ...,
    loggers=[file_logger]
)

Example output:

[FIT][step=2]: { "logged_metric": "logged_value", }
[EPOCH][step=2]: { "logged_metric": "logged_value", }
[BATCH][step=2]: { "logged_metric": "logged_value", }
[EPOCH][step=3]: { "logged_metric": "logged_value", }

Parameters

• filename (str, optional) –

  Format string for the filename.

  The following format variables are available:

  Variable	Description
  {run_name}	The name of the training run. See run_name.
  {rank}	The global rank, as returned by get_global_rank().
  {local_rank}	The local rank of the process, as returned by get_local_rank().
  {world_size}	The world size, as returned by get_world_size().
  {local_world_size}	The local world size, as returned by get_local_world_size().
  {node_rank}	The node rank, as returned by get_node_rank().

  Note

  When training with multiple devices (i.e. GPUs), ensure that '{rank}' appears in the format. Otherwise, multiple processes may attempt to write to the same file.

  Consider the following example when using default value of ‘{run_name}/logs-rank{rank}.txt’:

  >>> file_logger = FileLogger(filename='{run_name}/logs-rank{rank}.txt')
  >>> trainer = Trainer(loggers=[file_logger], run_name='my-awesome-run')
  >>> file_logger.filename
  'my-awesome-run/logs-rank0.txt'
  

  Default: ‘{run_name}/logs-rank{rank}.txt’

• artifact_name (str, optional) –

  Format string for the logfile’s artifact name.

  The logfile will be periodically logged (according to the flush_interval) as a file artifact. The artifact name will be determined by this format string.

  See also

  log_file_artifact() for file artifact logging.

  The same format variables for filename are available. Setting this parameter to None (the default) will use the same format string as filename. It is sometimes helpful to deviate from this default. For example, when filename contains an absolute path, it is recommended to set this parameter explicitely, so the absolute path does not appear in any artifact stores.

  Leading slashes ('/') will be stripped.

  Default: None (which uses the same format string as filename)

• capture_stdout (bool, optional) – Whether to include the stdout``in ``filename. (default: True)

• capture_stderr (bool, optional) – Whether to include the stderr``in ``filename. (default: True)

• buffer_size (int, optional) – Buffer size. See open(). Default: 1 for line buffering.

• log_level (LogLevel, optional) – LogLevel (i.e. unit of resolution) at which to record. Default: EPOCH.

• log_interval (int, optional) – Frequency to print logs. If log_level is EPOCH, logs will only be recorded every n epochs. If log_level is BATCH, logs will be printed every n batches. Otherwise, if log_level is FIT, this parameter is ignored, as calls at the FIT log level are always recorded. Default: 1.

• flush_interval (int, optional) – How frequently to flush the log to the file, relative to the log_level. For example, if the log_level is EPOCH, then the logfile will be flushed every n epochs. If the log_level is BATCH, then the logfile will be flushed every n batches. Default: 100.

• overwrite (bool, optional) – Whether to overwrite an existing logfile. (default: False)

property artifact_name

The artifact name for the logfile.

property filename

The filename for the logfile.

write(prefix, s)

Write to the logfile.

Note

If the write occurs before the INIT event, the write will be enqueued, as the file is not yet open.

Parameters

• prefix (str) – A prefix for each line in the logfile.

• s (str) – The string to write. Each line will be prefixed with prefix.