thygate / stable-diffusion-webui-depthmap-script

High Resolution Depth Maps for Stable Diffusion WebUI

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Video framerate depthmap/heatmap not matching original file + no codec?

eyeEmotion opened this issue · comments

I think these 2 might be related.

First problem is, the output is an avi file, which isn't even recognised by Microsoft's Video Editor, nor even professional ones like Davinci Resolve. So I have to convert them myself to a mp4, with a program like Handbrake.
And it is here that I don't know where the problem stems from. With now being an mp4, I can import it in my video editor, to use as a deptmap to convert my 2d movie to 3d. But I notice there as been atleast 1 frame is cut, which cause the frames not to line up at a certain point. And since I have to cut the video in 3 minute parts, that's quite a gap that is being created.
Because I have to convert it manually and am not able to test the avi, no editor is able to import it, I don't know if the problem already occurs when the depthmap/heatmap is generated or when I do the conversion from avi to mp4.

The video is also 23.976 fps, which I'm wondering if the depthmap generator can handle.
Also tried rerendering the movie to 24fps, but it's always outputted as 23.976fps.

So it seems to me that the original framerate of the video isn't taken into consideration as also the lack of a codec. Which also generated big files. The 3 minute avi is at around 800Mb, while with mp4 it's reduced to 8Mb.

Hello! Won't have time to look into it more, unfortunately. I would be happy if somebody looks into it and creates a fix.

The big file that you can not open is likely the depthmap video. It is a 16-bit monochrome video, you can try opening it in VLC. Please note that chopping the bit-depth to something like 8 will significantly reduce the depthmap quality and is not recommended for anything except for simply viewing it.

Hope this helps somehow.

Hi,

The past few days I've been trying to figure out what is wrong. Good news is, it isn't your Depthmap extenstion. Was able to go frame by frame on the AVI with LosslessCut and compare it with the imported converted MP4 in Davinci Resolve. And the AVI in fact had the frames that are missing when playing it in Resolve. I've even checked the converted MP4 and that one also has the missing frames.
My knowledge about codecs and such are just basic, so I don't know what is the issue with the video-editor cutting frames from the video.
I know Davinci Resolve can handle 16-bit format.

So if need be, I'll close this issue, but it would still be better to have a file that is immediatly useable in a video-editor, as that is where a depthmap video will probably get the most use.

Cheers.

Don't know if I can still reply to this. But I was still researching a bit regarding this problem and stumbled onto something you said.

So the AVI this Depthmap extension processes, is 16-bit monochrome, right? Now when I look in the Media Pool in Davinci Resolve, I notice the converted MP4 is 8 bit. Might that be the reason why Davinci Resolve cuts away frames? Because of the 16 to 8 bit?

Now I looked at Handbrake, and there is no 16 bit option to convert. The highest I see is 12 bit. So how do I convert a 16 bit AVI to a 16 bit MP4?