ReaJason / xhs

基于小红书 Web 端进行的请求封装。https://reajason.github.io/xhs/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

docker中崩溃

baixing99 opened this issue · comments

当请求量大的时候部署在docker中的签名服务端会崩溃容器挂掉


Exception occurred during processing of request from ('本机IP, 35690)
Traceback (most recent call last):
File "/usr/lib/python3.10/socketserver.py", line 316, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python3.10/socketserver.py", line 697, in process_request
t.start()
File "/usr/lib/python3.10/threading.py", line 940, in start
self._started.wait()
File "src/gevent/event.py", line 163, in gevent._gevent_cevent.Event.wait
File "src/gevent/_abstract_linkable.py", line 521, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait
File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
File "src/gevent/_abstract_linkable.py", line 451, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
File "/usr/local/lib/python3.10/dist-packages/playwright/sync_api/_context_manager.py", line 56, in greenlet_main
self._loop.run_until_complete(self._connection.run_as_sync())
File "/usr/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
self.run_forever()
File "/usr/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
self._run_once()
File "/usr/lib/python3.10/asyncio/base_events.py", line 1871, in _run_once
event_list = self._selector.select(timeout)
File "/usr/local/lib/python3.10/dist-packages/gevent/selectors.py", line 201, in select
self._ready.wait(timeout)
File "src/gevent/event.py", line 163, in gevent._gevent_cevent.Event.wait
File "src/gevent/_abstract_linkable.py", line 521, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait
File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
File "src/gevent/_abstract_linkable.py", line 455, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
gevent.exceptions.InvalidSwitchError: Invalid switch into Event.wait(): ()

Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/dist-packages/flask/main.py", line 3, in
main()
File "/usr/local/lib/python3.10/dist-packages/flask/cli.py", line 1064, in main
cli.main()
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 92, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/flask/cli.py", line 924, in run_command
run_simple(
File "/usr/local/lib/python3.10/dist-packages/werkzeug/serving.py", line 1109, in run_simple
srv.serve_forever()
File "/usr/local/lib/python3.10/dist-packages/werkzeug/serving.py", line 806, in serve_forever
super().serve_forever(poll_interval=poll_interval)
File "/usr/lib/python3.10/socketserver.py", line 232, in serve_forever
ready = selector.select(poll_interval)
File "/usr/lib/python3.10/selectors.py", line 416, in select
fd_event_list = self._selector.poll(timeout)
File "/usr/local/lib/python3.10/dist-packages/gevent/select.py", line 339, in poll
result.event.wait(timeout=timeout)
File "src/gevent/event.py", line 163, in gevent._gevent_cevent.Event.wait
File "src/gevent/_abstract_linkable.py", line 521, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait
File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
File "src/gevent/_abstract_linkable.py", line 455, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
gevent.exceptions.InvalidSwitchError: Invalid switch into Event.wait(): ()

同样的问题,你解决了吗?

请问是一个 client 一个账号在用这一个签名服务端导致的吗?

我是多个client,多账号在用一个签名服务端导致的

这个签名服务端有一个机制就是检测到 a1 不同的话会刷新界面导致资源消耗过大,我建议多个账号 cookie 里面设置成同样的 a1,至于一个 client 的情况就只能慢一点了,这个签名方式太垃圾了,不支持太快

def sign(uri, data, a1, web_session):
    global global_a1
    if a1 != global_a1: # 检测到 a1 不一样就刷新界面
        browser_context.add_cookies([
            {'name': 'a1', 'value': a1, 'domain': ".xiaohongshu.com", 'path': "/"}
        ])
        context_page.reload() # 刷新界面
        time.sleep(1)
        global_a1 = a1
    encrypt_params = context_page.evaluate("([url, data]) => window._webmsxyw(url, data)", [uri, data])
    return {
        "x-s": encrypt_params["X-s"],
        "x-t": str(encrypt_params["X-t"])
    }

我是多个client,多账号在用一个签名服务端导致的

问题解决了吗,同样是多个账号用一个签名服务器

我是多个client,多账号在用一个签名服务端导致的

问题解决了吗,同样是多个账号用一个签名服务器

还没解决,你可以试试启多个签名服务器

我是多个client,多账号在用一个签名服务端导致的

问题解决了吗,同样是多个账号用一个签名服务器

还没解决,你可以试试启多个签名服务器

好的。 是起多个docker, 使用不同的端口是吧。