composer.metrics

A collection of common torchmetrics.

Classes

BinaryF1Score	Implements F1 Scores for binary classification tasks via sklearn.
CrossEntropy	Torchmetrics cross entropy loss implementation.
Dice	The Dice Coefficient for evaluating image segmentation.
HFCrossEntropy	Hugging Face compatible cross entropy loss.
InContextLearningLMAccuracy	Computes accuracy for In-context learning (ICL) language modeling (LM) tasks.
InContextLearningLMExpectedCalibrationError	Computes Expected Calibration Error (ECE) for In-context learning (ICL) language modeling (LM) tasks.
InContextLearningMCExpectedCalibrationError	Computes Expected Calibration Error (ECE) for In-context learning (ICL) multiple choice (MC) tasks.
InContextLearningMetric
InContextLearningMultipleChoiceAccuracy	Computes accuracy for In-context learning (ICL) multiple choice (MC) tasks.
InContextLearningQAAccuracy	Computes accuracy for In-context learning (ICL) question answering (QA) tasks.
LanguageCrossEntropy	Torchmetric that computes cross entropy on language modeling outputs.
LanguagePerplexity	Subclasses LanguageCrossEntropy to implement perplexity.
LossMetric	Turns a torch.nn Loss Module into distributed torchmetrics Metric.
MAP	Computes the Mean-Average-Precision (mAP) and Mean-Average-Recall (mAR) for object detection predictions.
MIoU	Torchmetrics mean Intersection-over-Union (mIoU) implementation.
MaskedAccuracy	Computes accuracy with support for masked indices.
Perplexity	Subclasses HFCrossEntropy to implement perplexity.