Cannot read contract with hex literal

[Krzysztof Wędrowicz]

Hi,
I'd like to flatten contracts, from which one is OraclizeAPI.sol from OraclizeIT. I found that hex literals (like this below) throw the flattener:

bytes memory CODEHASH = hex"fd94fa71bc0ba10d39d464d0d8f465efeef0a2764e3887fcc9df41ded20f505c";

Error trace:

Traceback (most recent call last):
  File "/usr/local/bin/solidity_flattener", line 99, in <module>
    main()
  File "/usr/local/bin/solidity_flattener", line 94, in main
    solc_proc = subprocess.run(solc_args, stdout=subprocess.PIPE, universal_newlines=True)
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 405, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 830, in communicate
    stdout = self.stdout.read()
  File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 619797: invalid start byte

[Jitendra Chittoda]

I am also getting the same problem. I also wanted to fatten the Oraclize API in my contract.
Please fix this issue ASAP. Or suggest a workaround.

[Derrick Tan]

The problem is that the output of solc includes hex\"fd.., which is trying to interpret \"fd as a unicode character. In 3.6 subprocess added errors and encoding so errors="backslashreplace" should work. I tested locally with contents = open(sys.argv[1], 'r', errors="backslashreplace").read() and it worked fine with an oraclize lib.

[DarDev]

How to apply this hack?

[Mehmet Doğan]

How to apply this hack?

You need to make change on subprocess lib. The path of subprocess can be seen in traceback log and it should be something like ...../lib/python3.7/subprocess.py . Then replace init function as follows :
def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=True, shell=False, cwd=None, env=None, universal_newlines=None, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, **errors="backslashreplace"**, text=None):

Thanks to @dbtan for work-around