Facing Issue with Custom Model Integration

[Pypow]

Environment

OS: Windows 11
Unity Version: Unity 2021.3.19f1

Describe the bug
I need some help in integrating a custom MoViNet model in SSD scene, this is to work with another custom detection model (I have already integrated detection model).
As for the MoViNet model, I tried the approach mentioned in #320. I created copy of SSD.cs and BaseImagePredictor.cs and tweaked some code.

This model (as shown in the below screenshot) has multiple inputs & outputs out of which I am only interested in:
Input: location: 55 tensor: float32[1,1,172,172,3]) -> video input
Output: location: 1663 tensor: float32[1,2] -> class index + confidence score

Netron visualization of the model is shown below,

I am having trouble configuring SSD.cs for this model, as shown in the code snippet below, I believe I am making a mistake in configuration, maybe in initializing the variables MAX_DETECTION and outputs13. I get a Index Out of Bounds Exception or Tensorflow lite Failed when I change the values of these.

   ` const int MAX_DETECTION = 2;        //total no of classes
    private readonly float[,] outputs13 = new float[1,MAX_DETECTION]; // Throws Exception
    private readonly Result[] results = new Result[MAX_DETECTION];

    public override void Invoke(Texture inputTex)
    {
        ToTensor(inputTex, inputTensor);

        interpreter.SetInputTensorData(55, inputTensor);
        interpreter.Invoke();
        interpreter.GetOutputTensorData(13, outputs13); `

Could you please help me with this?

Thanks for your amazing work!

MoViNet_model.zip

[Koki Ibukuro]

Hi @Pypow

MoViNet utilizes a slightly different model input/output called signature runner, reusing the previous states of the MoViNet model. Please check out my MoViNet VideoClassification Example for the usage of the signature-runner.

(I don't have time to look at your model right now. Sorry if your model is not the signature model.)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.