[Bug]: Unable to refresh subscription, even with a VPN

Question

[Bug]: Unable to refresh subscription, even with a VPN

Froggy232 opened this issue 4 months ago · comments

Froggy232 commented 4 months ago

I've read the documentation

I'm running the latest version of Tube Archivist and have read the release notes.
I have read the how to open an issue guide, particularly the bug report section.

Operating System

Fedora Silverblue 39 + Docker

Your Bug Report

Describe the bug

Since few days ago, I can't refresh my subscriptions. I was thinking I was blocked from youtube, but even when I place tubearchivist behind airvpn, I get the same problem, and the exact same message in the logs.

Steps To Reproduce

Just try to refresh the subscriptions.

Expected behavior

It should refresh subscriptions.

Relevant log output

[2024-02-20 01:14:19,592: INFO/MainProcess] Task update_subscribed[9828ada6-6fe3-4a27-b6aa-6404654e916f] received
[2024-02-20 01:14:19,593: WARNING/ForkPoolWorker-32] update_subscribed create callback
[2024-02-20 01:14:19,654: WARNING/ForkPoolWorker-32] UCWc3I_Jt_YV3qtEvtvNOs-A: find missing videos.
[2024-02-20 01:14:39,757: WARNING/ForkPoolWorker-32] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f420f08ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution). Retrying (1/3)...
[2024-02-20 01:14:59,775: WARNING/ForkPoolWorker-32] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f420f08d710>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution). Retrying (2/3)...
[task][9828ada6-6fe3-4a27-b6aa-6404654e916f]: received STOP signal.
[2024-02-20 01:15:19,797: WARNING/ForkPoolWorker-32] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f420eed5bd0>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution). Retrying (3/3)...
[2024-02-20 01:15:39,814: WARNING/ForkPoolWorker-32] WARNING: [youtube:tab] Unable to download webpage: <urllib3.connection.HTTPSConnection object at 0x7f420eed7250>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution) (caused by TransportError("<urllib3.connection.HTTPSConnection object at 0x7f420eed7250>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution)")). Giving up after 3 retries
[2024-02-20 01:15:59,837: WARNING/ForkPoolWorker-32] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f420ef1c990>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution). Retrying (1/3)...
[2024-02-20 01:16:19,859: WARNING/ForkPoolWorker-32] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f420ef1dfd0>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution). Retrying (2/3)...
[2024-02-20 01:16:39,881: WARNING/ForkPoolWorker-32] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f420ef1d250>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution). Retrying (3/3)...
[2024-02-20 01:16:59,909: WARNING/ForkPoolWorker-32] ERROR: [youtube:tab] UCWc3I_Jt_YV3qtEvtvNOs-A: Unable to download API page: <urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution) (caused by TransportError("<urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution)"))
[2024-02-20 01:16:59,909: WARNING/ForkPoolWorker-32] https://www.youtube.com/channel/UCWc3I_Jt_YV3qtEvtvNOs-A/videos: failed to get info from youtube with message ERROR: [youtube:tab] UCWc3I_Jt_YV3qtEvtvNOs-A: Unable to download API page: <urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution) (caused by TransportError("<urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution)"))
[2024-02-20 01:16:59,913: WARNING/ForkPoolWorker-32] 9828ada6-6fe3-4a27-b6aa-6404654e916f Failed callback
[2024-02-20 01:16:59,914: ERROR/ForkPoolWorker-32] Task update_subscribed[9828ada6-6fe3-4a27-b6aa-6404654e916f] raised unexpected: ConnectionError('lost the internet, abort!')
Traceback (most recent call last):
  File "/root/.local/lib/python3.11/site-packages/urllib3/connection.py", line 203, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/socket.py", line 962, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -3] Temporary failure in name resolution

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/networking/_requests.py", line 306, in _send
    requests_res = session.request(
                   ^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/urllib3/util/retry.py", line 445, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/root/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/root/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/root/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1096, in _validate_conn
    conn.connect()
  File "/root/.local/lib/python3.11/site-packages/urllib3/connection.py", line 611, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/urllib3/connection.py", line 210, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 850, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 4082, in urlopen
    return self._request_director.send(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/networking/common.py", line 114, in send
    response = handler.send(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/networking/_helper.py", line 204, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/networking/common.py", line 325, in send
    return self._send(request)
           ^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/networking/_requests.py", line 334, in _send
    raise TransportError(cause=e) from e
yt_dlp.networking.exceptions.TransportError: <urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1587, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1722, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 718, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 4621, in wrapper
    info_dict = func(self, url, smuggled_data)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 6522, in _real_extract
    data, ytcfg = self._extract_data(url, display_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 5333, in _extract_data
    data = self._extract_tab_endpoint(url, item_id, ytcfg, fatal=fatal, default_client=default_client)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 5338, in _extract_tab_endpoint
    resolve_response = self._extract_response(
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 966, in _extract_response
    next(main_retries)
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/utils/_utils.py", line 5046, in __iter__
    self.error_callback(self.error, self.attempt, self.retries)
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 3767, in _error_or_warning
    RetryManager.report_retry(
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/utils/_utils.py", line 5054, in report_retry
    raise e
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 955, in _extract_response
    response = self._call_api(
               ^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 603, in _call_api
    return self._download_json(
           ^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1072, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1036, in download_handle
    res = self._download_webpage_handle(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 906, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 863, in _request_webpage
    raise ExtractorError(errmsg, cause=err)
yt_dlp.utils.ExtractorError: [youtube:tab] UCWc3I_Jt_YV3qtEvtvNOs-A: Unable to download API page: <urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution) (caused by TransportError("<urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/home/src/download/yt_dlp_base.py", line 64, in extract
    response = yt_dlp.YoutubeDL(self.obs).extract_info(url)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1576, in extract_info
    return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1605, in wrapper
    self.report_error(str(e), e.format_traceback())
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1054, in report_error
    self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
  File "/root/.local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 993, in trouble
    raise DownloadError(message, exc_info)
yt_dlp.utils.DownloadError: ERROR: [youtube:tab] UCWc3I_Jt_YV3qtEvtvNOs-A: Unable to download API page: <urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution) (caused by TransportError("<urllib3.connection.HTTPSConnection object at 0x7f420ef1ff50>: Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution)"))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.local/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/home/tasks.py", line 186, in update_subscribed
    missing_videos = handler.scan()
                     ^^^^^^^^^^^^^^
  File "/app/home/src/download/subscriptions.py", line 293, in scan
    self.scan_channels()
  File "/app/home/src/download/subscriptions.py", line 302, in scan_channels
    missing = channel_handler.find_missing()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/home/src/download/subscriptions.py", line 118, in find_missing
    last_videos = self.get_last_youtube_videos(channel_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/home/src/download/subscriptions.py", line 57, in get_last_youtube_videos
    channel = YtWrap(obs, self.config).extract(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/home/src/download/yt_dlp_base.py", line 77, in extract
    raise ConnectionError("lost the internet, abort!") from err
ConnectionError: lost the internet, abort!
[2024-02-20 01:16:59,916: WARNING/ForkPoolWorker-32] 9828ada6-6fe3-4a27-b6aa-6404654e916f return callback
[2024-02-20 01:18:42,429: INFO/MainProcess] Task update_subscribed[c4cdce1d-5a52-4dc5-bace-74a3e5f622c1] received
[2024-02-20 01:18:42,430: WARNING/ForkPoolWorker-32] update_subscribed create callback
[2024-02-20 01:18:42,482: WARNING/ForkPoolWorker-32] UCWc3I_Jt_YV3qtEvtvNOs-A: find missing videos.

Anything else?

Thanks for your help!

Simon · Answer 1 · Wed Feb 21 2024 03:43:22 GMT+0800 (China Standard Time)

Says right there at the bottom:

ConnectionError: lost the internet, abort!

You don't have internet.

Froggy232 · Answer 2 · Wed Feb 21 2024 03:53:14 GMT+0800 (China Standard Time)

Thanks for your answer, but when I try to open a console in the container and do a curl, everything work.
I will try to remove everything and restart from the beginning.
Thanks for your help

Simon · Answer 3 · Wed Feb 21 2024 04:22:55 GMT+0800 (China Standard Time)

Or maybe no DNS.

Froggy232 · Answer 4 · Wed Feb 21 2024 09:11:22 GMT+0800 (China Standard Time)

Yes, but why it would not work only for tubearchivist?
As I said, if I try a curl in the container, it works.

Jayden · Answer 5 · Sat Feb 24 2024 09:42:14 GMT+0800 (China Standard Time)

Yes, but why it would not work only for tubearchivist? As I said, if I try a curl in the container, it works.

Your logs suggest no DNS resolution to youtube.com:
Failed to resolve 'www.youtube.com' ([Errno -3] Temporary failure in name resolution)

If you attach to the containers shell can you ping youtube.com?

Simon · Answer 6 · Sun Mar 10 2024 20:38:19 GMT+0800 (China Standard Time)

Assuming this has been solved by now.