Error: Response is constant across permutations. Please check your model

[Jiaxiang Li]

LIME seizes me, because of its good display way.
I try to use LIME to explain Xgboost Model. for text classification topic, I find there are a lot of successful minimal examples online, such as this one.
I reproduce it successfully.

However, when I apply lime functions on a Chinese-character dataset, I always get this error message.

Error: Response is constant across permutations. Please check your model

It seemingly means that the target variable is constant. I double check it on the origin function.
See it here

But in my dataset, my target variable is a binary variable instead of a constant one.

For reproducible purpose, I upload the dataset and minimal exmaple code here.

1. bad_comment.csv
2. good_comment.csv
3. stopwords.txt
4. lime-issue.R

The last code of .R file produces the error in the title.

[Jiaxiang Li]

I find this kind of error results from the structure input.
Here is my test results.

test_comment_1 <- "卧槽，这酒也太好喝了，简直没的说，绝对的物有所值，下来再买"
lime::explain(test_comment_1,explainer,n_labels = 1,n_features = 5) %>% plot_features

I get the successful output.

But the next one,

test_comment_2 <- "红红火火"
safely(lime::explain)(test_comment_2,explainer,n_labels = 1,n_features = 5)

## $result
## NULL
## 
## $error
## <simpleError: Response is constant across permutations. Please check your model>

I think the input includes duplicated words 红 and 火.

[BrazilForever11]

I got the same issue, I am guessing this is due to class imbalance. When you pick value from the test set, lime finds neighborhood around this value in train set and using permutation picks value from this neighborhood. And due to class imbalance all values come out to be of the same class, hence the error.