AI/deep learning upscale of movies

[eddebaby]

If any one wants to try upscaling the video frames, here they all are ("extracted" with ffmpeg): https://drive.google.com/file/d/1n96Np6CdP9NGAz-GC5I4yuWNwjc8DArt/view?usp=sharing.

This is probably a good resource for AI upscaling of Game resources: https://upscale.wiki/wiki/Main_Page#ESRGAN
(it seems to talk about training a deep learning algorithm).

Getting the results back in to KeeperFX will likely need some coding, but there is no point doing this unless good results can be achieved with the upscaling!

Contents of Video Proprieties.txt:

Video properties (from ffmpeg, could be wrong I guess):

bullfrog - Duration: 00:00:09.50, Dimensions: 640x240, Framerate: 20 fps
drag     - Duration: 00:00:05.68, Dimensions: 320x200, Framerate: 14.08 fps
outromix - Duration: 00:00:37.91, Dimensions: 320x200, Framerate: 17.54 fps
intromix - Duration: 00:01:56.91, Dimensions: 320x200, Framerate: 17.54 fps

bullfrog.smk is interlaced when played by the game, and plays at an effective 640x480.

drag has no audio, otherwise the audio is included as .wav files (PCM 22050 Hz / 16bit)
intromix and outromix are stereo, bullfrog is mono

Example ffmpeg commands:

Extract video frames :-

ffmpeg -i drag.smk drag/drag_%d.png
ffmpeg -i bullfrog.smk bullfrog/bullfrog_%d.png
ffmpeg -i intromix.smk intromix/intromix_%d.png
ffmpeg -i outromix.smk outromix/outromix_%d.png

Extract audio as .wav :-
[junk video is created to ensure audio length matches video length]

ffmpeg -i intromix.smk -map 0:a intromix.wav -map 0:v junk.avi
ffmpeg -i outromix.smk -map 0:a outromix.wav -map 0:v junk.avi
ffmpeg -i drag.smk -map 0:a drag.wav -map 0:v junk.avi
ffmpeg -i bullfrog.smk -map 0:a bullfrog.wav -map 0:v junk.avi

Create test MKV files from wav and png files :-

ffmpeg -framerate 17.54 -i intromix/intromix_%d.png -i intromix.wav -c:v libx264 -crf 0 -c:a aac intromix.mkv
ffmpeg -framerate 17.54 -i outromix/outromix_%d.png -i outromix.wav -c:v libx264 -crf 0 -c:a aac outromix.mkv
ffmpeg -framerate 20 -i bullfrog/bullfrog_%d.png -i bullfrog.wav -vf scale=iw:ih*2  -c:v libx264 -crf 0 -c:a aac bullfrog.mkv
ffmpeg -framerate 14.08 -i drag/drag_%d.png -c:v libx264 -crf 0 -c:a aac drag.mkv