Redundant lines
ayyucedemirbas opened this issue · comments
You already called the standardize() function in the tokenize() function.
def tokenize(self, text):
text = self.standardize(text)
return text.split()
Is it necessary to call it again before the tokenize() function call on the following lines?
Thank you so much!