Distinct-N
Distinct-N, most notably distinct-1 and distinct-2, is a metric that measures the diversity of a sentence. It focuses on the number of distinct n-grams of a sentence and thus penalizes sentences with lots of repeated words. The metric is free of any reference or ground truth sentence and devotes totally to the property of a sentence (generated by the system). It is proposed by Jiwei Li et.al in the paper A Diversity-Promoting Objective Function for Neural Conversation Models.
Definitions
The original paper coined Distinct-N as:
We report the degree of diversity by calculating the number of distinct unigrams and bigrams in generated responses.
The value is scaled by the total number of generated tokens to avoid favoring long sentences
which is exactly what we have mentioned before.
Dependencies
python>=3.6.1
References
[1] A Diversity-Promoting Objective Function for Neural Conversation Models