tgxs002 / HPSv2

Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Can I use it to compare different text-image pairs?

Duanener opened this issue · comments

Hi, thanks for sharing such great work. However, there is a question.
If I only have a bunch of image and text pairs, that is, [[prompt1, image1], [prompt2, image2], ...[promptN, imageN]], there is a one-to-one correspondence between them, not a one-to-many relationship, may I ask? , in this case how should I use your model to rank the aesthetic and human preference scores of these image-text pairs?

Theoretically, this is not recommended, because our training data takes the form of single prompt vs. multiple images. The training target can not guarantee a good comparison between images of different prompts.
But if you try ranking images of different prompts, you can get some meaningful result. Here is an example: https://tgxs002.github.io/hps_filter.github.io/ . If you are just interested in filtering a dataset, you can give it a try, and see if it works for you.

Thanks for your reply.