Unbabel / COMET

A Neural Framework for MT Evaluation

https://unbabel.github.io/COMET/html/index.html

Unbabel/COMET Issues

Add missing library stubs or py.typed marker
Updated a month ago
[QUESTION] Comet kiwi architecture
Updated a month ago11
[QUESTION] OOM when load XCOMET-XXL in A100 with 40G memory for prediction
Updated a month ago4
Training data and scripts used for wmt22-cometkiwi-da
Updated a month ago3
[QUESTION] why num_layers = num_hidden_layers + 1
Closed a month ago1
Training script for XCOMET
Closed a month ago1
Safetensors Support
Updated 2 months ago
Change the global variable logger to comet_logger
Updated 2 months ago1
TypeError: comet.models.utils.Prediction is not a dataclasss. This is a subclass of ModelOutput and so must use the @dataclass decorator.
Closed 8 months ago7
[INPUT] Text Length of Input (source, reference, and hypothesis)
Updated 2 months ago2
[QUESTION] Memory footprint
Closed 2 months ago21
504 Server error when running comet-score using multiple machines
Updated 2 months ago9
[QUESTION] large file scoring
Closed 3 months ago3
[QUESTION] Splitting big models over multiple GPUs
Updated 3 months ago6
[QUESTION] Keep getting scores of '0' no matter what input used
Closed 3 months ago5
[QUESTION] Train UnifiedMetric/XCOMET with word level predictions.
Updated 3 months ago1
Different scores from different COMET package versions 1.1.2 and 2.2.1
Updated 3 months ago2
Runtime error when loading wmt23-cometkiwi-da-xl
Closed 3 months ago1
if tgt is same with src, the score is still high
Closed 3 months ago2
Different versions of COMET code give different scores with the same model and date.
Closed 3 months ago
what is the precision when load_from_checkpoint?
Updated 4 months ago
Minimizing cpu RAM vs only use GPU RAM
Updated 4 months ago1
Invalid link reference of reference-free model in readme
Closed 4 months ago
HF Hub dependency conflicts with latest `tranformers` version.
Closed 5 months ago3
Sparsemax not actually used in COMET-KIWI, XCOMET-XL/XXL
Updated 5 months ago4
[QUESTION] Unable to use ReLU as activation function
Closed 5 months ago4
AttributeError: 'dict' object has no attribute 'scores'
Closed 5 months ago2
Mismatch between error span and offsets
Closed 5 months ago2
Multi-gpu inference returns AttributeError
Closed 5 months ago7
[QUESTION] Reliability for en-th and th-en pair
Closed 6 months ago2
AttributeError: 'dict' object has no attribute 'scores' when using the model 'Unbabel/wmt22-cometkiwi-da'
Closed 7 months ago4
When loading huggingface Unbabel/wmt22-cometkiwi-da checkpoint, UnpicklingError
Closed 6 months ago
[QUESTION] AttributeError: An error occurred during model prediction: 'dict' object has no attribute 'scores'
Closed 6 months ago5
[QUESTION] Train Your Own Metric
Updated 7 months ago2
Comet Evaluation Metric Error with Command line Utility
Closed 7 months ago1
whar aspects of language gets considered while evaluating the tranlsation by COMET. as far as I am aware of MQM metric sare covered in it. Interested to know other terminolofy/features of language gets in COMET.
Closed 7 months ago1
what aspects of language get considered while evaluating the translation by COMET. as far as I am aware of MQM metrics are covered in it. Interested in knowing other terminology/features of language gets in COMET. I am looking for metric along with COMET that can assess the translated content for cosmetic domain data. please suggest.
Updated 7 months ago1
Running the code on Mac
Closed 8 months ago
Not compatible with recent transformers?
Closed 8 months ago3
Quantization
Updated 8 months ago2
Multi-GPU training
Closed 8 months ago10
Is `wmt22-comet-da` the same as "COMET-22" & trained on MQM data?
Closed 9 months ago2
Using comet-mbr for Multi-Model Translation Ranking: Questions About Input Format and GPU Disabling
Updated 10 months ago3
[QUESTION] Why does training speed go down?
Updated 10 months ago5
pretrained_model setting in hparams.yaml has no effect
Closed 10 months ago1
[QUESTION]When I train my COMET model, I have the following problem when I am almost successful, it seems to be stuck
Closed 10 months ago2
Unbabel/wmt22-cometkiwi-da doesn't work from Huggingface
Closed a year ago1
v1.x and v2.x have different scores for wmt20-comet-qe-da model
Closed a year ago2
[QUESTION] How are the older models supposed to be used?
Closed a year ago
[QUESTION] Does COMET support Scoring multiple refs like scarebleu?
Closed a year ago2