composer.Engine
The order of algorithms can matter significantly during composition. For example, the Selective Backprop algorithm runs during AFTER_DATALOADER
event, and must run before any data augmentations. The Engine
runs these re-ordering passes.
Note
The design of the Engine
will be changed in future releases to accomdate more complexity as we investigation the composition of algorithms.
Currently, the following passes are registered:
LIFO order for events
For events that follow the
after_*
andbefore_*
pattern, the ordering of algorithms is reversed for theafter_*
events. For example, algorithms will run in aABCD -> DCBA
ordering before and after say, the loss computation.This allows algorithms to “clean up” their changes. e.g. Label smoothing will smooth the labels upon entry to the loss, and then restore the original unsmoothed labels upon exit.
Run Selective Backprop first
Selective backprop runs after the dataloader returns the batch, and executes an extra forward pass to rank and prune the examples in the batch by loss. To ensure a clean estimate of the example, Selective backprop should run before any other data augmentations during
AFTER_DATALOADER
(e.g. such as MixUp).
- class composer.Engine(state: State, algorithms: Sequence[Algorithm], logger: Optional[Logger] = None, callbacks: Optional[Sequence[Callback]] = None)[source]
Coordinator for running and resolving conflicts between algorithms.
- Parameters
state (State) – the initial
State
of the trainer. Will be modified in-place.algorithms (Sequence[Algorithm]) – the list of algorithms for this engine to execute.
logger (Optional[Logger]) – a
Logger
instance to be used for logging algorithm and callback specific metrics.callbacks (Sequence[Callback]) – the list of callbacks for this engine to execute.
- run_event(event: Union[Event, str]) Dict[str, Trace] [source]
Runs the sequence of algorithms and callbacks.
Filters algorithms by calling each one’s
match()
function, internally checks for conflicting algorithms, then runs each algorithm’sapply()
function to make in-place changes to theState
.The order of algorithm execution is determined by the provided list, plus any changes made internally to prevent conflicts.
Returns traces of the execution, a dictionary with keys formatted as
<algorithm_name>/<event>
(e.g.Blurpool/TRAINING_START
), and values are thecomposer.core.engine.Trace
object, which include an optional return code from the algorithm, the order of execution, and whether the algorithm was run.Callbacks are always ran after algorithms, and do not return a trace.
Can be called with either the Event enum, or a string of the event value.
Examples
>>> engine = Engine(state, algorithms, logger, callbacks) >>> engine.run_event(Event.BEFORE_LOSS) # or >>> engine.run_event('before_loss') # also works
Trace
Traces record whether an algorithm ran at a particular step and event combination, and also the order of such executions. These are logged with the key {algorithm_name}/{event}
.
For example, the algorithm Layer Freezing
, which runs at the end of every epoch, will emit a series of traces:
[STEP=0][layer_freezing/TRAINING_START=0]
[STEP=1][layer_freezing/EPOCH_START=0]
[STEP=1][layer_freezing/BATCH_START=0]
...
[STEP=2][layer_freezing/BATCH_START=0]
...
[STEP=3][layer_freezing/BATCH_START=0]
...
[STEP=3][layer_freezing/EPOCH_END=1] # <-- ran here!