A troubling python script-commands execution problem

[zhlongiter]

I am writing a python script command to format the mybatis log, replacing the placeholders in the sql with specific query parameters. This script will read the contents of clipboard and parse it. Here is my python code

#!/usr/bin/env python3

# Required parameters:
# @raycast.schemaVersion 1
# @raycast.title parse
# @raycast.mode fullOutput

# Optional parameters:
# @raycast.icon 🦈
# @raycast.packageName parse mybatis log

# Documentation:
# @raycast.description 解析mybatis的日志
# @raycast.author zhlong
# @raycast.authorURL https://raycast.com/zhlong

import re
import subprocess
import pyperclip


def color_text(text, color_code):
    return f"\033[{color_code}m{text}\033[0m"


def parse_sql():
    # 获取粘贴板中的内容
    # log_text = subprocess.check_output(["pbpaste"], universal_newlines=True)
    log_text = pyperclip.paste()

    # 匹配 SQL 语句和参数部分的正则表达式
    sql_pattern = re.compile(r".*Preparing: (.+)")
    param_pattern = re.compile(r".*Parameters: (.+)")

    # 从日志中提取 SQL 语句和参数
    sql_match = sql_pattern.search(log_text)
    param_match = param_pattern.search(log_text)

    if sql_match and param_match:
        sql_statement = sql_match.group(1).strip()
        params = param_match.group(1).split(", ")

        # 检查params是否为空
        if params:
            # 替换 SQL 语句中的占位符
            for param in params:
                if param == "null":
                    param_value = "null"
                else:
                    param_value, param_type = (
                        param.split("(")[0].strip(),
                        param.split("(")[1][:-1].strip(),
                    )  # 提取参数值和类型
                    # 如果日期时间格式为"YYYY-MM-DDThh:mm:ss"，则替换"T"为空格
                    if re.match(r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}", param_value):
                        param_value = re.sub("T", " ", param_value)
                    if param_type == "String":
                        param_value = (
                            "'" + param_value + "'"
                        )  # 为 String 类型参数添加单引号
                sql_statement = sql_statement.replace("?", param_value, 1)

        pyperclip.copy(sql_statement)  # 写入粘贴板

        print(color_text(sql_statement, "36"))
        print(color_text("\nSQL语句已复制到粘贴板", "31"))
    else:
        print(color_text("No SQL or parameters found in the log.", "33"))


if __name__ == "__main__":
    parse_sql()

The problem is that if the content in my clipboard has Chinese characters, for example

Preparing: select * from table_a where state = ?
2024-05-07 09:28:36,885 DEBUG [XNIO-1 task-2] [e593cd8f-2f97-4b68-aae8-279a7bd7bd43] d.c.q.a.d.w.d.M.listDippingTaskCloth [BaseJdbcLogger.java : 137] ==> Parameters: 正常(String)
2024-05-07 09:28:36,886 DEBUG [XNIO-1 task-2] [e593cd8f-2f97-4b68-aae8-279a7bd7bd43] d.c.q.a.d.w.d.M.listDippingTaskCloth [BaseJdbcLogger.java : 137] <==      Total: 0

When this command is executed through raycast, the following exception message appears

  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyperclip/__init__.py", line 127, in paste_osx_pbcopy
    return stdout.decode(ENCODING)
           ^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 215: invalid continuation byte

However, if i set the log_text vaue with the log,like this:

    log_text = """
        Preparing: select t.prod0task0code , m.label0code from ent_haiyangkejigufenyoux_datapool_mesppre0prd0tsk0info t join ent_haiyangkejigufenyoux_datapool_mesf0task0material m on t.prod0task0code = m.prod0task0code and m.feed0no0num > 0 and m.material0type = '白布' and m.feed0type = '自动备料' and m.disable != 1 where t.resource0code = ? and t.production0state in('未开始','进行中') and t.disable != 1 order by t.order0num + 0 asc, m.order0num + 0 asc
        2024-05-07 09:28:36,885 DEBUG [XNIO-1 task-2] [e593cd8f-2f97-4b68-aae8-279a7bd7bd43] d.c.q.a.d.w.d.M.listDippingTaskCloth [BaseJdbcLogger.java : 137] ==> Parameters: EQ-202403-00004(String)
        2024-05-07 09:28:36,886 DEBUG [XNIO-1 task-2] [e593cd8f-2f97-4b68-aae8-279a7bd7bd43] d.c.q.a.d.w.d.M.listDippingTaskCloth [BaseJdbcLogger.java : 137] <==      Total: 0
    """

and execute again ,there is no exception!!!

[Marcos Sánchez-Dehesa]

Hey there @zhlongiter,

Sorry it took a while to answer this. Sadly, I am not very knowledgeable about Python. I can only tell you that we run the scripts as a subprocess passing to the environment the following variables:

• LANG. The value is whatever your locale preferred identifier is (in my case en_US.UTF-8, notice I do live in Europe though, but this is my preferred system locale).
• PATH. We prepend /usr/local/bin:/opt/homebrew/bin to your system $PATH.
• Proxy settings if any (in Raycast settings). Such as http_proxy, https_proxy, no_proxy, etc.

You should try to print the environment variables from your python script and see whether changing the values there make any difference.

[Marcos Sánchez-Dehesa]

I will close this for now. Feel free to reopen whenever you want.