Explicitly set encoding when calling open

[matthewh]

The long dashes in README.rst raise a UnicodeDecodeError when building the project on Python 3.5.2. Unfortunately python 3.5.2 is the default python 3 version in Ubuntu 16.04 LTS.

root@ba3058541282:/src/syncrepl# pip3 wheel .
Processing /src/syncrepl
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-1v94ry17-build/setup.py", line 103, in <module>
        long_description = readme(),
      File "/tmp/pip-1v94ry17-build/setup.py", line 95, in readme
        return file.read()
      File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1706: ordinal not in range(128)

Specifying the encoding when opening the file fixes the issue

def readme():
    with open('README.rst', encoding='utf8') as file:
        return file.read()

[A. Karl Kornel]

Hello!

I've got an Ubuntu 16.04 system, and I'm not able to reproduce the issue as you described it. See the following asciicast:

https://asciinema.org/a/DG8xAbfQt0kVs99TyHEiOapZG

I wonder, what output do you get from the locale command? If your locale is set to something like C or US-ASCII, then that would explain the error you're getting.

[A. Karl Kornel]

Looks like two files need to be changed: setup.py calls open in two locations; docs/conf.py calls open once. Both should have encodings hard-coded.

[matthewh]

Hello @akkornel,

I build all of my python projects using docker. The image in question is based on the official docker xenial one with a few python libraries added. Good call on checking locale, that isn't something I've set in the past.

docker run -it --rm  multipart/python3-dev
root@7a68891575fd:/# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

A quick google search reveals I'm not the first person to be bit by this type of issue!