ezyang / ghstack

Submit stacked diffs to GitHub on the command line

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ghstack crashing nondeterministically

hsghori opened this issue · comments

I have a stack of >10 PRs and ghstack seems to be crashing with an ambiguous error:

ERROR: Fatal exception
Traceback (most recent call last):
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/home/hsghori/.pyenv/versions/3.9.11/lib/python3.9/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/home/hsghori/.pyenv/versions/3.9.11/lib/python3.9/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/home/hsghori/.pyenv/versions/3.9.11/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/xxxx (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/ghstack/logs.py", line 105, in manager
    yield
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/ghstack/cli.py", line 41, in cli_context
    yield shell, config, github
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/ghstack/cli.py", line 207, in submit
    ghstack.submit.main(
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/ghstack/submit.py", line 220, in main
    submitter.push_updates()
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/ghstack/submit.py", line 1058, in push_updates
    self.github.patch(
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/ghstack/github.py", line 61, in patch
    return self.rest("patch", path, **kwargs)
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/ghstack/github_real.py", line 125, in rest
    resp: requests.Response = getattr(requests, method)(
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/requests/api.py", line 145, in patch
    return request("patch", url, data=data, **kwargs)
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/home/hsghori/.local/pipx/venvs/ghstack/lib/python3.9/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/xxxx (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))

I think this is the thing where if your stack is too long we do too many queries to GH API and we get throttled. The only workaround is to have less diffs in your stack atm

duping as #121