Omar-ALkhateeb / Misfaia

An Open Source API to help filter arabic text and check for toxicity and/or offensiveness

Misfaia

the Arabic toxicity filter API

An Open Source API to help filter arabic text and check for toxicity and/or offensiveness

aknowledgements

• dataset used Arabic-twitter-corpus-AJGT-
• inspired by every other bad word filter and the lack of ararbic one's (as far as i know)

todo

• create basic model with +90% accuracy
  • fixed model serialization issue
  • add pruning and quantization for faster model ref1 ref2
• wrap around an api for ease of use
  • add CORS
  • jwt-auth
  • rate-limiter
  • security headers
• add a bad words filter
  • update curses list to a txt/csv file
  • fill the dictionary
• ingestion engine?

routes

• /removeCurses --- bad word filter
• /getPrecentage --- AI
• /cleanCorpus --- remove punctuation and other chars