''utf-8'' codec can''t decode byte 0x82 in position XX: invalid start byte'

[Karthick31]

Stackstorm winrm_ps_script action failure to stdout the non ascii characters
st2 version - 3.8.0
OS environment - RHEL 7

[root@server1 actions]# st2 run default.Ad_user_validation
..
id: 652788a0aec49d1752a2aaf0
action.ref: default.Ad_user_validation
context.user: st2admin
parameters:
password: '********'
username: svc_user
status: failed
start_timestamp: Thu, 12 Oct 2023 05:48:16 UTC
end_timestamp: Thu, 12 Oct 2023 05:48:18 UTC
result:
error: '''utf-8'' codec can''t decode byte 0x82 in position 36: invalid start byte'
traceback: " File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2actions/container/base.py", line 132, in _do_run
(status, result, context) = runner.run(action_params)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/winrm_runner/winrm_ps_script_runner.py", line 47, in run
return self.run_ps(ps_script, ps_params)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/winrm_runner/winrm_base.py", line 381, in run_ps
return self._run_ps(encoded_ps, is_b64=True)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/winrm_runner/winrm_base.py", line 396, in _run_ps
return self._translate_response(response)
File "/opt/stackstorm/st2/lib/python3.6/site-packages/winrm_runner/winrm_base.py", line 261, in _translate_response
result["stdout"] = result["stdout"].decode("utf-8")
"

[AJ]

This looks like old 2.7 code. remove the .decode and you should be good to go.

[AJ]

if this is in st2 core, we will need to fix it actually.

[Karthick31]

@guzzijones - Could you please elaborate the steps. On which file, .decode should be removed ?

[StealthCT]

The issue is, that logic only runs when the response is six.binary_type (or bytes today). The responses from pywinrm are binary strings, so they definitely need converted/decoded, but the response likely isn't UTF-8.

I note in pywinrm's test runner, that they define a unicode round trip test: https://github.com/diyan/pywinrm/blob/f796b5aa15f0ce6c3e16aa3fd33a13efedff4937/winrm/tests/test_integration_protocol.py#L8C41-L8C55

There they define the codepage as 65001 (which is Windows for UTF-8):

shell_id = protocol_real.open_shell(codepage=65001)

..with the default codepage being 437. Considering we have a custom function here to extend what pywinrm offers as arguments, we should consider setting this to 65001 so the binary response is actually UTF-8 and can be decoded as such.

[Karthick31]

Thanks @Stealthii.

Replaced the below code in winrm_base.py & issue is fixed.

< shell_id = session.protocol.open_shell(env_vars=env, working_directory=cwd, codepage=65001)

    shell_id = session.protocol.open_shell(env_vars=env, working_directory=cwd)