Does llm-guard's text content detection technology support languages other than English, such as Chinese?
dongxinghua opened this issue · comments
Hey @dongxinghua ,
thanks for raising this! In theory, some of these checks should work with Chinese but we don't guarantee that. It needs some testing, and we are planning to work on multi-language features in the coming months.
Best regards,
Oleksandr
@dongxinghua I have discovered llm-guard
this week and I like to add this to one of our projects. I also bumped into question similar as yours (but Filipino instead of Chinese language). One work-around I came up with is to implement an input scanner following the implementation of Toxicity
class.
https://github.com/laiyer-ai/llm-guard/blob/main/llm_guard/input_scanners/toxicity.py
You can swap with existing or your own hugging face models to support other languages or other use cases that you may have.