type of dataset
Hossein-1991 opened this issue · comments
Hossein Salahshoor Gavalan commented
Hi,
My question is kind of basic!
I would like to use keybert, but I don't know whether removing punctuations are helpful or not!
More deeply, are punctuations essential for text classification tasks?
Maarten Grootendorst commented
It highly depends on the embedding model that you use. For most transformer-based models, it is important to keep the punctuations as they are part of the context.