composer.utils.batch_helpers#

composer.utils.batch_helpers

Functions

`batch_get`	Indexes into the batch given the key.
`batch_set`	Indexes into the batch given the key and sets the element at that index to value.

composer.utils.batch_helpers.batch_get(batch, key=None, get_fn=None)[source]#

Indexes into the batch given the key.

>>> from composer.utils.batch_helpers import batch_get
>>> batch_get([1,2,3], 1)
2
>>> batch_get({'a':1, 'b':7}, 'b')
7
>>> batch_get([{'a':1, 'b':7},{'c':5}], get_fn=lambda x: x[1]['c'])
5

Parameters

batch (Any) – An object that contains the input and label of the items in the batch. Can be any abritrary type that user creates, but we assume some sort of sequence (list, tuple, tensor, array), mapping (dictionary), or attribute store (object with data members, namedtuple).
key (Any) – A key to index into the batch. Key is optional if get_fn is supplied.
get_fn (Callable) – A user-specified function to do the extracting. get_fn is optional if key is supplied.

Returns

The part of the batch specified by the key or the get_fn. This could be any type – depending on what the batch is composed of.

Raises

ValueError if key is unset and get_fn is unset or if both are set. –

composer.utils.batch_helpers.batch_set(batch, *, key=None, value, set_fn=None)[source]#

Indexes into the batch given the key and sets the element at that index to value.

This is not an in-place operation for batches of type tuple as tuples are not mutable.

>>> from composer.utils.batch_helpers import batch_set
>>> batch_set([1,2,3], key=1, value=8)
[1, 8, 3]
>>> batch_set({'a':1, 'b':7}, key='b', value=11)
{'a': 1, 'b': 11}
>>> def setter(batch, value):
...     batch[1]['d'] = value
...     return batch
...
>>> batch_set([{'a':1, 'b':7},{'d':3}], value=20, set_fn=setter)
[{'a': 1, 'b': 7}, {'d': 20}]

Parameters

batch (Any) – An object that contains the input and label of the items in the batch. Can be any abritrary type that user creates, but we assume some sort of sequence (list, tuple, tensor, array), mapping (dictionary), or attribute store (object with data members, namedtuple).
key (Any) – A key to index into the batch.
value (Any) – The value that batch[key] or batch.key gets set to.
set_fn (Callable) – A user-specified function to do the setting. set_fn is optional if key and value are supplied. set_fn must return the updated batch.

Returns

batch (Any) – updated batch with value set at key.

Raises

ValueError if –

key and set_fn are both unset * key and set_fn are both set