Lightning-AI / torchmetrics

Torchmetrics - Machine learning metrics for distributed, scalable PyTorch applications.

https://lightning.ai/docs/torchmetrics/

Lightning-AI/torchmetrics Issues

Binary Classification Expected Calibration Error
Closed 9 days ago2
List Metric synchronization fails in corner case
Closed 16 days ago1
Ordinal classification metrics
Updated 19 days ago1
Behaviors of AUROC and Average Precision are inconsistent when all labels are equal
Closed 21 days ago4
Can't access metrics in a MetricCollection via keys returned in MetricCollection.keys
Closed 21 days ago
`list` states leak (`Tensor`) memory
Closed 22 days ago1
Metrics not being logged properly on remote GPU
Updated 22 days ago4
Error in ERGAS metric
Closed 23 days ago5
Bug in ERGAS
Closed 23 days ago1
BootStrapper.update/forward don't process kwargs
Closed 23 days ago1
Mean Average Detection ignores `warn_on_many_detections` set to False
Closed 23 days ago1
Building the docs
Closed 24 days ago2
The default value of `compute_with_cache` should be `True` .
Closed 25 days ago2
Specificity if TN + FN = 0
Closed a month ago1
Incorrect result in computing `MulticlassRecall` macro average when `ignore_index` is specified
Updated a month ago1
Contribution: Add new audio/speech metrics for generative audio
Updated a month ago4
Metric not moved to device and invalids the cpu-gpu offloading when combining with DeepSpeed
Updated a month ago1
[Bug] No backend type associated with device type cpu
Updated a month ago1
Retrieval Metrics GPU Memory Leak
Updated a month ago1
Discrepancy in optimal threshold calculation between sklearn and torchmetrics ROC implementations
Updated a month ago2
Create gallery of realistic examples
Updated a month ago
Add Support for SQ and RQ in Panoptic Quality
Closed a month ago5
MulticlassAveragePrecision crashes on .compute() if empty
Closed a month ago2
`log_dict` method breaks on `MultitaskWrapper` + `MetricCollection` combination
Closed 2 months ago2
ClasswiseWrapper and JaccardIndex confmat attribute error
Closed 2 months ago2
Unable to plot `MetricCollection` containing prefix using `MetricCollection.plot()`
Closed 2 months ago1
Error during argument validation: predictions can not contain `ignore_index`
Closed 2 months ago1
Incorrect value of AUROC when plotting a `PrecisionRecallCurve` metric with `score=True`
Closed 2 months ago2
RetrievalNormalizedDCG doesn't change with different top_k values
Closed 2 months ago2
Add a CW-SSIM support for torchmetrics
Updated 2 months ago8
Delay imports of optional dependencies such as torchaudio, torchvision
Closed 2 months ago1
Wrong aggregation of Precision\Recall\F1-Score
Updated 2 months ago2
A way to run *any* metric async on cpu
Updated 2 months ago1
Typing error for detection metrics in MultitaskWrapper
Closed 2 months ago2
Importing torchmetrics causes segmentation fault with other dependencies
Closed 2 months ago2
top-k multiclass macro accuracy is not calculated correctly
Closed 2 months ago
`MetricCollection` did not copy inner state of metric in `ClasswiseWrapper` when computing groups metrics
Closed 2 months ago1
top-k multiclass macro accuracy less than top-1 multiclass macro accuracy
Closed 2 months ago2
A new metric NPV
Updated 2 months ago1
`MultitaskWrapper` still cannot be logged
Closed 3 months ago1
MPS uninitialized memory(?) causing errors in `StatScores` (which cascade to other locations)
Updated 3 months ago1
Explicitly initializing tensors in `float32` in MeanMetric goes against `torch.set_default_dtype`, leading to numerical errors
Closed 3 months ago6
Add an option to switch distributions order in the KLDivergence.
Closed 3 months ago2
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)
Closed 3 months ago2
MetricWrapper for Target Binarization
Updated 3 months ago3
Total sum of squares formula
Closed 3 months ago3
MeanAveragePrecision doesn't work as expected when using `max_detection_thresholds != [1, 10, 100]`
Closed 3 months ago3
Custom callable is ignored in retrieval metric aggregation
Closed 3 months ago1
High memory usage of Perplexity metric
Closed 3 months ago2
usage consistency on GPU between `MetricCollection` and `FeatureShare`
Closed 3 months ago1