mohdsm81 / ytdlp-wrapper-scripts

wrapper scripts for the ytdlp command line util for consistently downloading and renaming files, as well as recoding them to MP4 to avoid transcoding from media servers for the most part.

Readme

These scripts can be placed or linked (i.e. ln -s download-video /usr/local/bin/download-video) to one of your PATH directories.

It works and was tested with the fork of youtube-dl called yt-dlp. The latter has additional features and is actively maintained/updated.

Additionally, if you want to add more flags/options when calling any of the scripts, you can do it like the following (assuming the list has say 60+ videos):

download-list https://some.awesome.list.com --playlist-start 53 --playlist-end 60

Another way to specify items exolicitly to download from a playlist is the following (assuming it has 31 items or more):

download-list https://some.awesome.list.com --playlist-items 31,27

And so on… Have fun!

Scheduling tips and streamlining

Create list on youtube/rumble with the visibility set to “unlisted” (which is not private nore public but any one with a link can access it). Instead of adding videos to the “Watch Later” list, add it to this unlisted list. This way you don’t need any authentication, and no one knows about it.

Now, on your local machine, on which these videos are to be downloaded, you can schedule a task to invoke the cd <<destination-folder>> && download-list-auto <<my-unlisted-list-url>> in crontab to run every minute or what have you. Once you add a video to that list, within a max of 1 minute (in my case) it will start downloading it.

So, you only need to add videos to the list, and it gets downloaded. This script won’t take a lot of CPU or redownload what has been. This is because it keeps track of what has been downloaded in the local folder to that <<destination-folder>>/downloaded.txt list of hashes.

Caveat

You will need to delete the files from the folder manually. Maybe yt-dlp has an option to remove files not in the list any more, but I don’t want to use it… because once the videos are downloaded, I want to empty the list, and add few more videos to be downloaded, but that doesn’t mean I want my videos (potentially unwatched yet) be deleted from my folder.

My final setup stub

Following this great guide about how to use flock, I could use the scripts on a cronjob that runs every minute without launching infinite amount of processes that would end up messing up the download folder and potentially corrupting files.

This is the stub (fill in your specific info between the pair of << and >> and remove these angle brackets):

* * * * * /usr/bin/flock -w 0 /var/lock/yt-dlp/download-list.lock cd <<destination-folder>> && \
    /usr/local/bin/download-list-auto <<my-unlisted-list-url>> \
                                 > /var/log/download-list.log 2>&1

PLEASE NOTE: I am using download-list-auto for automation and download-list for manual downloading of lists. Reason being that everytime I remove or add a video to my to-watch list, it re downloads all those videos with different index in the playlist, which leads to duplication. Now I removed the index and playlist name from the auto version of the list to avoid this confusion.

I hope this helps!

Additional scripts for channel-wide downloads

I added a do.organize.sh script so that once you use the download-list on a whole channel, i.e. downloading the entire playlists folder for example, it will download all playlists into one root folder. That is the folder where you invoked the script.

Now, the root folder will contain all downloaded files in a flat structured manner. You can then invoke the do.organize.sh script in the root folder. What this script does is:

1. iterate over all files and extract the playlists names
2. create folders for each playlist
3. move all files belonging to each playlist to its appropriate folder.
4. move all folders with their content, and the downloaded.txt to the created channel folder.

NOTE: When trying to update a list, in order to avoid trying to download all downloaded files, you need to use --download-archive ../downloaded.txt (i.e. the one residing in the channel’s root folder) while invoking the download-list script from within each playlist folder. Why? because the downloaded.txt contains all playlists’ files that has been downloaded and need to be updated with newly downloaded files.

How to use yank.channel.sh script

Follow the steps:

1. Navigate to he channel, then you click on the playlists tab
2. copy the URL, this is what you will pass to the script as a CLI argument
3. you navigate to a root folder of your media folders where this channel is to have a folder of its own.
4. run the script, passing the URL above to it. Once done downloading, the script will call the do.organize.sh script to do the remaining steps of re-organizing the channel folder and files.
5. Once the download finishes, all the necessary folders (corresponding to each playlist) will be created and all files belonging to each playlist will be placed in the corresponding folder.
6. Next, the script will move all created folders to the channel-named folder, and will leave the downloaded.txt file beside the channel’s folder. DO NOT move it, it will become relevant in later steps.
7. Now, suppose everything goes perfectly, good the script did its intended purpose.
8. Suppose there has been updates to the channel’s playlist(s) or the script didn’t download all files in the playlists of that channel. Then you do the following:
   1. Navigate to the root folder where the channel’s folder resides along side the downloaded.txt file. (soon this will be clear why)
   2. invoke the same script with the same url. The script will use the downloaded.txt file to skip all those downloaded files and download only the one’s not marked as completed/downloaded in the text file.
   3. Once done downloading, it will move the newly downloaded folder content to the appropriate folder residing inside the channel’s folder.
   4. Do this to update the channel’s downloaded files

Not Tested Scripts Yet

1. =yank.channel.sh=, now tested and works fine, so far.

Developer notes

These notes are intended for those who want to modify the scripts, improve them, or add error handling, … etc.

yt-dlp error codes

Taken from this issue/post:

  0 = Everything is successful
100 = yt-dlp must restart for update to complete
101 = Download cancelled by --max-downloads etc
  2 = Error in user-provided options
  1 = Any other error