Too much info from `WARNING | util | UnicodeError`
meanrin opened this issue · comments
Running scanner on repo with binary files can easily transform CLI out to
2021-12-07 16:17:11,870 | WARNING | util | UnicodeError: Can't read content from "/mnt/data/datasets/validate_repos/node/test/fixtures/wpt/encoding/legacy-mb-tchinese/big5/big5_errors.html" as utf16.
2021-12-07 16:17:11,903 | WARNING | util | UnicodeError: Can't read content from "/mnt/data/datasets/validate_repos/node/test/fixtures/wpt/encoding/legacy-mb-tchinese/big5/big5_chars-csbig5.html" as utf8.
2021-12-07 16:17:11,903 | WARNING | util | UnicodeError: Can't read content from "/mnt/data/datasets/validate_repos/node/test/fixtures/wpt/encoding/legacy-mb-tchinese/big5/big5_chars-csbig5.html" as utf16.
2021-12-07 16:17:11,918 | WARNING | util | UnicodeError: Can't read content from "/mnt/data/datasets/validate_repos/node/test/wasi/wasm/create_symlink.wasm" as utf8.
2021-12-07 16:17:11,930 | WARNING | util | UnicodeError: Can't read content from "/mnt/data/datasets/validate_repos/node/test/wasi/wasm/create_symlink.wasm" as utf16.
2021-12-07 16:17:11,960 | WARNING | util | UnicodeError: Can't read content from "/mnt/data/datasets/validate_repos/node/test/fixtures/wpt/encoding/resources/two-boms-utf-16be.html" as utf8.
2021-12-07 16:17:11,960 | WARNING | util | UnicodeError: Can't read content from "/mnt/data/datasets/validate_repos/node/test/fixtures/wpt/encoding/resources/two-boms-utf-16le.html" as utf8.
2021-12-07 16:17:11,964 | WARNING | util | UnicodeError: Can't read content from "/mnt/data/datasets/validate_repos/node/test/fixtures/wpt/encoding/resources/utf-32-big-endian-bom.html" as utf8.
This is not as useful, especially due to the fact that utf-16 files would always create this log message even thou they would be read correctly
Propose to change this log level to INFO or DEBUG
CredSweeper/credsweeper/utils/util.py
Line 98 in 7c0b7e9
so it would not be spammed during the scan with default (warning) log level
I agree to change the log level too.