OnroerendErfgoed / language-tags

Dealing with IANA language tags in Python

Home Page:https://pypi.python.org/pypi/language-tags

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Bug under windows

koenedaele opened this issue · comments

From OnroerendErfgoed/skosprovider#16, as reported by it seems there's a problem with language-tags. When opening the json files, Python tries to guess the encoding the file is in. On some (or all) Windows systems, this ends up being cp-1252. While the file is actually in utf-8. This causes errors like:

File "D:\skosp\ENV\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 31946: ch
aracter maps to <undefined>