processors/github_metadata: retry in case of API request timeout
nodiscc opened this issue · comments
Would prevent having to manually re-run the whole job, e.g. https://github.com/awesome-selfhosted/awesome-selfhosted-data/actions/runs/5662427414/job/15342323076
[...]
INFO:github_metadata.py: Gathering metadata for https://github.com/revive-adserver/revive-adserver from Github API
INFO:github_metadata.py: Gathering metadata for https://github.com/revoltchat/revolt from Github API
INFO:github_metadata.py: Gathering metadata for https://github.com/roadiz/roadiz from Github API
INFO:github_metadata.py: Gathering metadata for https://github.com/robustirc/robustirc from Github API
INFO:github_metadata.py: Gathering metadata for https://github.com/RocketChat/Rocket.Chat from Github API
Traceback (most recent call last):
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 536, in _make_request
response = conn.getresponse()
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 461, in getresponse
httplib_response = super().getresponse()
File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
response.begin()
File "/usr/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.10/socket.py", line 705, in readinto
return self._sock.recv_into(b)
File "/usr/lib/python3.10/ssl.py", line 1274, in recv_into
return self.read(nbytes, buffer)
File "/usr/lib/python3.10/ssl.py", line 1130, in read
return self._sslobj.read(len, buffer)
TimeoutError: The read operation timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen
retries = retries.increment(
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 470, in increment
raise reraise(type(error), error, _stacktrace)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/urllib3/util/util.py", line 39, in reraise
raise value
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
response = self._make_request(
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 538, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.github.com', port=443): Read timed out. (read timeout=15)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/bin/hecat", line 8, in <module>
sys.exit(main())
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/hecat/main.py", line 39, in main
add_github_metadata(step)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/hecat/processors/github_metadata.py", line 107, in add_github_metadata
gh_metadata, latest_commit_date = get_gh_metadata(step, github_url, g, errors)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/hecat/processors/github_metadata.py", line 59, in get_gh_metadata
gh_metadata = g.get_repo(project)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/github/MainClass.py", line 363, in get_repo
headers, data = self.__requester.requestJsonAndCheck("GET", url)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/github/Requester.py", line 443, in requestJsonAndCheck
*self.requestJson(
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/github/Requester.py", line 580, in requestJson
return self.__requestEncode(cnx, verb, url, parameters, headers, input, encode)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/github/Requester.py", line 685, in __requestEncode
status, responseHeaders, output = self.__requestRaw(
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/github/Requester.py", line 719, in __requestRaw
response = cnx.getresponse()
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/github/Requester.py", line 164, in getresponse
r = verb(
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/home/runner/work/awesome-selfhosted-data/awesome-selfhosted-data/.venv/lib/python3.10/site-packages/requests/adapters.py", line 532, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.github.com', port=443): Read timed out. (read timeout=15)
make: *** [Makefile:25: update_metadata] Error 1
Error: Process completed with exit code 2.