composer.metrics#

A collection of common torchmetrics.

Classes

BinaryF1Score

Implements F1 Scores for binary classification tasks via sklearn.

CrossEntropy

Torchmetrics cross entropy loss implementation.

Dice

The Dice Coefficient for evaluating image segmentation.

HFCrossEntropy

Hugging Face compatible cross entropy loss.

InContextLearningLMAccuracy

Computes accuracy for In-context learning (ICL) language modeling (LM) tasks.

InContextLearningMetric

InContextLearningMultipleChoiceAccuracy

Computes accuracy for In-context learning (ICL) multiple choice (MC) tasks.

LanguageCrossEntropy

Torchmetric that computes cross entropy on language modeling outputs.

LossMetric

Turns a torch.nn Loss Module into distributed torchmetrics Metric.

MAP

Computes the Mean-Average-Precision (mAP) and Mean-Average-Recall (mAR) for object detection predictions.

MIoU

Torchmetrics mean Intersection-over-Union (mIoU) implementation.

MaskedAccuracy

Computes accuracy with support for masked indices.

Perplexity

Subclasses HFLanguageCrossEntropyLoss to implement perplexity.