[Bug]: No results when scanning for subscriptions
rafcon-dev opened this issue · comments
I've read the documentation
- I'm running the latest version of Tube Archivist and have read the release notes.
- I have read the how to open an issue guide, particularly the bug report section.
Operating System
Ubuntu Server
Your Bug Report
Hey. As asked on Discord (issue with same name), here I post on github.
Getting the dreaded unable to extract yt initial data. This started happening after the migration of last summer to the new filenames. It still complains about some files not being found also...
Basically, if I scan for subscriptions, nothing happens, in the logs, it can be seen it's something to do with yt-dlp. However, if log in onto the container and run yt-dlp, I can successfully download youtube videos.
Relevant log output
[T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataIndexTemplateService","elasticsearch.cluster.uuid":"1reV6Z87RFOfnMyoeAnGmA","elasticsearch.node.id":"v8cvjeCOS-urSjEPVSPHBg","elasticsearch.node.name":"163657f79e04","elasticsearch.cluster.name":"docker-cluster"}
archivist-es | {"@timestamp":"2024-02-16T21:39:23.956Z", "log.level": "INFO", "message":"updating index template [.monitoring-logstash-mb] for index patterns [.monitoring-logstash-8-*]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[163657f79e04][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.metadata.MetadataIndexTemplateService","elasticsearch.cluster.uuid":"1reV6Z87RFOfnMyoeAnGmA","elasticsearch.node.id":"v8cvjeCOS-urSjEPVSPHBg","elasticsearch.node.name":"163657f79e04","elasticsearch.cluster.name":"docker-cluster"}
tubearchivist | . thumbnail_check
tubearchivist | . update_subscribed
tubearchivist | . version_check
tubearchivist |
tubearchivist | [2024-02-16 21:39:31,548: WARNING/MainProcess] /root/.local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
tubearchivist | whether broker connection retries are made during startup in Celery 6.0 and above.
tubearchivist | If you wish to retain the existing behavior for retrying connections on startup,
tubearchivist | you should set broker_connection_retry_on_startup to True.
tubearchivist | warnings.warn(
tubearchivist |
tubearchivist | [2024-02-16 21:39:31,556: INFO/MainProcess] Connected to redis://archivist-redis:6379//
tubearchivist | [2024-02-16 21:39:31,557: WARNING/MainProcess] /root/.local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
tubearchivist | whether broker connection retries are made during startup in Celery 6.0 and above.
tubearchivist | If you wish to retain the existing behavior for retrying connections on startup,
tubearchivist | you should set broker_connection_retry_on_startup to True.
tubearchivist | warnings.warn(
tubearchivist |
tubearchivist | [2024-02-16 21:39:31,560: INFO/MainProcess] mingle: searching for neighbors
tubearchivist | [2024-02-16 21:39:32,570: INFO/MainProcess] mingle: all alone
tubearchivist | [2024-02-16 21:39:32,587: INFO/MainProcess] celery@89a5da5c764f ready.
tubearchivist | expire session in 172800 secs
tubearchivist | [2024-02-16 21:41:22,724: INFO/MainProcess] Task update_subscribed[08a13942-0a56-411a-bae3-3f092db2932d] received
tubearchivist | [2024-02-16 21:41:22,726: WARNING/ForkPoolWorker-8] update_subscribed create callback
tubearchivist | [2024-02-16 21:41:26,068: WARNING/ForkPoolWorker-8] UCYO_jab_esuFRV4b17AJtAw: find missing videos.
tubearchivist | [2024-02-16 21:41:26,777: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:26,778: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (1/3)...
tubearchivist | [2024-02-16 21:41:26,973: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:26,974: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (2/3)...
tubearchivist | [2024-02-16 21:41:27,164: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:27,164: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (3/3)...
tubearchivist | [2024-02-16 21:41:27,348: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:27,349: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U. Giving up after 3 retries
tubearchivist | [2024-02-16 21:41:27,353: WARNING/ForkPoolWorker-8] ERROR: [youtube:tab] UCYO_jab_esuFRV4b17AJtAw: Playlists that require authentication may not extract correctly without a successful webpage download. If you are not downloading private content, or your cookies are only for the first account and channel, pass "--extractor-args youtubetab:skip=authcheck" to skip this check
tubearchivist | [2024-02-16 21:41:27,353: WARNING/ForkPoolWorker-8] https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw/videos: failed to get info from youtube with message ERROR: [youtube:tab] UCYO_jab_esuFRV4b17AJtAw: Playlists that require authentication may not extract correctly without a successful webpage download. If you are not downloading private content, or your cookies are only for the first account and channel, pass "--extractor-args youtubetab:skip=authcheck" to skip this check
tubearchivist | [2024-02-16 21:41:27,735: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:27,735: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (1/3)...
tubearchivist | [2024-02-16 21:41:27,908: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:27,909: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (2/3)...
tubearchivist | [2024-02-16 21:41:28,096: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:28,096: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (3/3)...
tubearchivist | [2024-02-16 21:41:28,287: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:28,287: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U. Giving up after 3 retries
tubearchivist | [2024-02-16 21:41:28,289: WARNING/ForkPoolWorker-8] ERROR: [youtube:tab] UCYO_jab_esuFRV4b17AJtAw: Playlists that require authentication may not extract correctly without a successful webpage download. If you are not downloading private content, or your cookies are only for the first account and channel, pass "--extractor-args youtubetab:skip=authcheck" to skip this check
tubearchivist | [2024-02-16 21:41:28,289: WARNING/ForkPoolWorker-8] https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw/streams: failed to get info from youtube with message ERROR: [youtube:tab] UCYO_jab_esuFRV4b17AJtAw: Playlists that require authentication may not extract correctly without a successful webpage download. If you are not downloading private content, or your cookies are only for the first account and channel, pass "--extractor-args youtubetab:skip=authcheck" to skip this check
tubearchivist | [2024-02-16 21:41:28,677: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
tubearchivist | [2024-02-16 21:41:28,677: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (1/3)...
tubearchivist | [2024-02-16 21:41:28,857: WARNING/ForkPoolWorker-8] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
archivist-es | {"@timestamp":"2024-02-16T21:39:23.972Z", "log.level": "INFO", "message":"updating index template [.monitoring-kibana-mb] for index patterns [.monitoring-kibana-8-*]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[163657f79e04][masterService#updateTaskAnything else?
No response
Here is a full log
TAlogs.txt
You are probably getting blocked by YT. FAQ has some solutions. If you find other approaches that help, please add them there.
Beyond that, not much we can do here.
The weird thing is that I can use yt-dlp inside of the container. Shouldn't that get blocked too if that was the case?
The weird thing is that I can use yt-dlp inside of the container. Shouldn't that get blocked too if that was the case?
When I get blocked on Tube Archivist I can still watch YouTube on my TV, so it's not blocking the entire network. Maybe try clearing your cookie? Settings>application>disable cookie
Nobody except YT people knows how they monitor their things. What we can obserf and guess is they do some kind of environment pattern matching, IP is only one part of that. But, that's a guess, changing the IP breaks their pattern matching avoiding that limitation.
It indeed was just a matter of disabling the use of the cookie. Things seem to be working ok for now! Cheers!