"Chat Download" never stops
CappiSteijns opened this issue · comments
I'm not sure what is happening, but the "chat download" never stop, resulting in errors. Some downloads are successfully recorded and moved, while others are not. See the pictures and logs below.
The logs in ganymede itself;
"Video Download" log:
ganymede-api-2024-06-13T15-46-06.log
ganymede-temporal-2024-06-13T15-47-47.log
If you need more logs let me know please :)
EDIT: added docker compose
services:
ganymede-api:
container_name: ganymede-api
image: ghcr.io/zibbp/ganymede:latest
restart: unless-stopped
environment:
- TZ=Europe/Amsterdam # Set to your timezone
- DB_HOST=192.168.68.61
- DB_PORT=4803
- DB_USER=ganymede
- DB_PASS=####
- DB_NAME=ganymede-prd
- DB_SSL=disable
- JWT_SECRET=SECRET
- JWT_REFRESH_SECRET=SECRET
- TWITCH_CLIENT_ID=############xjm7o
- TWITCH_CLIENT_SECRET=6################or6j
- FRONTEND_HOST=https://###########.nl
- COOKIE_DOMAIN=########.nl
# OPTIONAL
# - OAUTH_PROVIDER_URL=
# - OAUTH_CLIENT_ID=
# - OAUTH_CLIENT_SECRET=
# - OAUTH_REDIRECT_URL=http://IP:PORT/api/v1/auth/oauth/callback # Points to the API service
# Volgende heb ik toegevoegd voor UPDATE 2.0.0
- TEMPORAL_URL=ganymede-temporal:7233
# WORKER
- MAX_CHAT_DOWNLOAD_EXECUTIONS=5
- MAX_CHAT_RENDER_EXECUTIONS=3
- MAX_VIDEO_DOWNLOAD_EXECUTIONS=5
- MAX_VIDEO_CONVERT_EXECUTIONS=3
volumes:
- /mnt/user/Twitch/ganymede:/vods
- /mnt/user/appdata/ganymede-api/logs:/logs
- /mnt/user/appdata/ganymede-api:/data
# Uncomment below to persist temp files
- /mnt/user/Twitch/ganymede/temp:/tmp
ports:
- 4800:4000
ganymede-frontend:
container_name: ganymede-frontend
image: ghcr.io/zibbp/ganymede-frontend:latest
restart: unless-stopped
environment:
- REQUIRE_LOGIN=true
- API_URL=https://apiganymede.cappiflex.nl # Points to the API service
- CDN_URL=https://cdnganymede.cappiflex.nl # Points to the CDN service
- SHOW_SSO_LOGIN_BUTTON=true # show/hide SSO login button on login page
- FORCE_SSO_AUTH=false # force SSO auth for all users (bypasses login page and redirects to SSO)
ports:
- 4801:3000
ganymede-db:
container_name: ganymede-db
image: postgres:14
volumes:
- /mnt/user/appdata/ganymede-db:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=#######
- POSTGRES_USER=ganymede
- POSTGRES_DB=ganymede-prd
ports:
- 4803:5432
ganymede-nginx:
container_name: ganymede-nginx
image: nginx
volumes:
- /mnt/user/appdata/ganymedenginx/nginx.conf:/etc/nginx/nginx.conf:ro
- /mnt/user/Twitch/ganymede:/mnt/vods
ports:
- 4802:8080
ganymede-temporal:
image: temporalio/auto-setup:1.23
container_name: ganymede-temporal
depends_on:
- ganymede-db
environment:
- DB=postgres12 # this tells temporal to use postgres (not the db name)
- DB_PORT=5432
- POSTGRES_USER=ganymede
- POSTGRES_PWD=#####
- POSTGRES_SEEDS=ganymede-db # name of the db service
ports:
- 7233:7233
It looks like a similar issue to #437 which I've pushed a fix for to the :main
image. When there is nothing being archived, can you switch the API container to use the :main
image and try again?
I plan to publish a release this weekend with some fixes that have been building up the past few weeks.
It looks like a similar issue to #437 which I've pushed a fix for to the
:main
image. When there is nothing being archived, can you switch the API container to use the:main
image and try again?I plan to publish a release this weekend with some fixes that have been building up the past few weeks.
Allrighty, changed to :main for the api container.
Any chance the vods in the queue will finish?
When going to Admin - Tasks - Start stuck queue items I get the error: Request Failed Key: 'StartTaskRequest.Task' Error:Field validation for 'Task' failed on the 'oneof' tag
If those are livestreams, there's not much you can do to recover the chats. If you have /tmp
mounted locally that chats will likely still be there. For those queue items, you'll have to delete them from Admin > Queue, and set the vod to "processed" in Admin > VODs.
The admin task is deprecated and doesn't work. It is removed, waiting on a release to push it out.
If those are livestreams, there's not much you can do to recover the chats. If you have
/tmp
mounted locally that chats will likely still be there. For those queue items, you'll have to delete them from Admin > Queue, and set the vod to "processed" in Admin > VODs. The admin task is deprecated and doesn't work. It is removed, waiting on a release to push it out.
So I have like all these files in the /tmp
folder:
and set the vod to "processed" in Admin > VODs.
When I do that the vod does become clickable in Ganymede but it won't load the vod because it's still in the /tmp
folder and does not get moved to the correct folder. What would be the steps to manual move the vod's and chat files from /tmp
? If that would even be possible.
You can try to manually restart the workflow items for each archive. Start by navigating to Workflows > Closed. Then you'll need to find the workflow you want to run and confirm that it corresponds with your video ID. Start clicking into workflows and finding that one that matches your video ID.
Once you've found the group of workflows for your video ID, locate the one that you want to "jumpstart". We'll start with the task after the chat download, ConvertTwitchLiveChatWorkflow
.
Clicking the restart button on the workflow page will restart the workflow. Hopefully this will take the *-live-chat.json
file and convert it into the standard format that can be used.
After the live chat has been converted, you'll need to find the MoveTwitchChatWorkflow
for your video ID and run that as well. Repeat the same for the incomplete video workflows as well.
The logs of the ganymede-api
container will tell you if this process is going smooth or if it's run into issues.
It's a complex process to get these going again so let me know if you have any questions/issues.
Hi, i seem to have to same issue. Recently migrated from 1.43 on a synology, to 2.3.0, now 2.3.1 on a windows machine.
I would really like to get at least one memorable vod archived.
A lot of them seem to fail on KillTwitchLiveChatDownload.
The problem is i cant seem to find ConvertTwitchLiveChatWorkflow, to "jumpstart" it.
All the -live-chat and -video files are in the tmp directory.
Also notice that start time and close time is "invalid date" on all closed workflows. Active workflows have invalid date on start time, but the current time on close time.
The problem is i cant seem to find ConvertTwitchLiveChatWorkflow, to "jumpstart" it.
All the -live-chat and -video files are in the tmp directory.
Ya that's right because the workflow didn't progress past the download. I can likely recover everything with some manual effort. First, make a copy of all the data in the Ganymede container to somewhere on your host to make sure it doesn't get deleted.
docker cp ganymede-api:/tmp ./tmp-backup
I'll walk through the manual archive steps below.
Video
To perform the manual steps of the video archive process, follow the below instructions.
- Exec into the ganymede-api container with
docker exec -it ganymede-api /bin/bash
. - Identify the video in
/tmp
that you want to go through the archive process with. For example,44359154267_d1283376-2b5c-11ef-b9de-0242ac1a0007-video.mp4
- Run the following ffmpeg command to perform some video post-processing, updating with your correct video path. Note the
-convert
appended to the output file name.
ffmpeg -i /tmp/44359154267_d1283376-2b5c-11ef-b9de-0242ac1a0007-video.mp4 -c:v copy -c:a copy /tmp/44359154267_d1283376-2b5c-11ef-b9de-0242ac1a0007-video-convert.mp4
- Edit the VOD in Ganymede (Admin > VODs) and copy the "Video Path".
- Copy the
-convert
file from the FFmpeg command to the path you copied
cp /tmp/44359154267_d1283376-2b5c-11ef-b9de-0242ac1a0007-video-convert.mp4 /vods/staysafetv/2024-06-15-44359154267-live-d1283376-2b5c-11ef-b9de-0242ac1a0007/44359154267-video.mp4
- Go back to editing the VOD in Ganymede and disable the "Is Processing" checkmark at the top and save. Now if you view the VOD you should be able to play the video.
Chat
Converting the chat is more difficult. If you send me the following information/files I can do this for you. I will need the following:
- The
-live-chat.json
files from the/tmp
directory - The
chat_start
time for each chat/VOD. You can retrieve this by running the below command. Update the ID with the Ganymede ID of the VOD you are recovering
docker exec ganymede-db psql -U ganymede ganymede-prd -c "select chat_start from queues where vod_queue = 'd1283376-2b5c-11ef-b9de-0242ac1a0007'"
v2.3.2 has been released that includes the fix to this chat download issue. If you're attempting to recover failed archives, I advise to hold off on updating until you get those recovered.
Hi, thank you so much!
Updated to v2.3.1:main tag on api and fronted just before you replied. I see that fixed the invalid dates.
I was able to convert and transfer the video files to the archive.
Here are the two chats:
Chat 1:
40658934565_49e946eb-1493-11ef-8dc9-0242ac150007-live-chat.json
chat_start
2024-05-17 21:20:32.790546+00
Chat 2:
42362888808_18a5b85a-1759-11ef-8dc9-0242ac150007-live-chat.json
chat_start
2024-05-21 10:01:33.420771+00
Here are the converted chat files. One of them is likely longer than the video as it failed to stop downloading. This shouldn't affect the playback though as the start time is what really matters. You'll need to move these files to the correct path found by editing the VOD in Ganymede and copying to the chat_path
.
40658934565_49e946eb-1493-11ef-8dc9-0242ac150007-chat-convert.json
42362888808_18a5b85a-1759-11ef-8dc9-0242ac150007-chat-convert.json
If you want to render these chats into videos for long-term archival reasons. Follow the below steps
- Exec into the ganymede-api container with
docker exec -it ganymede-api /bin/bash
. - Copy the chat path for each video from Ganymede
- Run the following command
TwitchDownloaderCLI chatrender -i <path_to_chat_json> <arguments> -o /tmp/chat.mp4
path_to_chat_json
is the path to the chat json you just copiedarguments
can be retrieved from Admin > Settings and copying the arguments from "Chat Render Args"
- Once the chat render is complete move the video to the correct location
If you're not trying to recover any more failed archives I recommend updated to the images to the latest version now.
Yeah i was able to successfully archive the chats!
The issue was only intermittent for me, and i haven't been able to test out v2.3.2 yet. Im sure you can close it, ill report back if it still a problem after some testing.