tensorflow / flutter-tflite

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Cannot get yamnet features in flutter

mikkel-convai opened this issue · comments

Hi

I am trying to use the yamnet tflite model from: "/kaggle/input/yamnet/tflite/tflite/1/1.tflite", to take in a 500 ms sound clip and extract embeddings and spectrogram.

I can load the model fine with:
final interpreter = await Interpreter.fromAsset('assets/1.tflite');

And get both input and output tensors like so:

interpreter.resizeInputTensor(0, singleSample.shape);
inputTensors = interpreter.getInputTensors();
Tensor inputTensor = inputTensors[0];
outputTensors = interpreter.getOutputTensors();
scores = outputTensors[0];
embeddings = outputTensors[1];
spectrogram = outputTensors[2];

The problem arises in figuring out how to format the input and output. I've tried the following:
First

  final input = List.generate(8000, (_) => List.filled(1, 0));
  final output = {0: scores, 1: embeddings, 2: spectrogram};
  interpreter.runForMultipleInputs(input, output);

Which gives error:

E/flutter (11660): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: RangeError (index): Invalid value: Only valid value is 0: 1
E/flutter (11660): #0 _Array.[] (dart:core-patch/array.dart:10:36)
E/flutter (11660): #1 ListMixin.elementAt (dart:collection/list.dart:78:33)
E/flutter (11660): #2 Interpreter.runInference
interpreter.dart:196
E/flutter (11660): #3 Interpreter.runForMultipleInputs
interpreter.dart:180
E/flutter (11660): #4 get_yamnet_features_single_sample
yamnet_features.dart:31

Second

final input = List.generate(8000, (_) => List.filled(1, 0));
 final output = {0: scores, 1: embeddings, 2: spectrogram};
 interpreter.run(input, output);

Which gives error:

E/tflite (11660): tensorflow/lite/kernels/pad.cc:79 SizeOfDimension(op_context->paddings, 0) != op_context->dims (1 != 2)
E/tflite (11660): Node number 15 (PAD) failed to invoke.
E/flutter (11660): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Bad state: failed precondition
E/flutter (11660): #0 checkState
check.dart:74
E/flutter (11660): #1 Interpreter.invoke
interpreter.dart:164
E/flutter (11660): #2 Interpreter.runInference
interpreter.dart:214
E/flutter (11660): #3 Interpreter.runForMultipleInputs
interpreter.dart:180
E/flutter (11660): #4 Interpreter.run
interpreter.dart:172
E/flutter (11660): #5 get_yamnet_features_single_sample
yamnet_features.dart:31
E/flutter (11660):

Third

  final input = List.generate(8000, (_) => List.filled(1, 0));
  final output = [scores, embeddings, spectrogram];
  interpreter.run(input, output);

Which gives error:

E/tflite (11660): tensorflow/lite/kernels/pad.cc:79 SizeOfDimension(op_context->paddings, 0) != op_context->dims (1 != 2)
E/tflite (11660): Node number 15 (PAD) failed to invoke.
E/flutter (11660): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Bad state: failed precondition
E/flutter (11660): #0 checkState
check.dart:74
E/flutter (11660): #1 Interpreter.invoke
interpreter.dart:164
E/flutter (11660): #2 Interpreter.runInference
interpreter.dart:214
E/flutter (11660): #3 Interpreter.runForMultipleInputs
interpreter.dart:180
E/flutter (11660): #4 Interpreter.run
interpreter.dart:172
E/flutter (11660): #5 get_yamnet_features_single_sample
yamnet_features.dart:31
E/flutter (11660):

Fourth

List<double> scoresList = List.filled(521, 0.0);
List<double> embeddingsList = List.filled(1024, 0.0);
List<double> spectrogramList = List.filled(64, 0.0);

final input = List.generate(8000, (_) => List.filled(1, 0));
final output = {0: scoresList, 1: embeddingsList, 2: spectrogramList};
interpreter.runForMultipleInputs(input, output);

Which again gives error:

E/flutter (11660): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: RangeError (index): Invalid value: Only valid value is 0: 1
E/flutter (11660): #0 _Array.[] (dart:core-patch/array.dart:10:36)
E/flutter (11660): #1 ListMixin.elementAt (dart:collection/list.dart:78:33)
E/flutter (11660): #2 Interpreter.runInference
interpreter.dart:196
E/flutter (11660): #3 Interpreter.runForMultipleInputs
interpreter.dart:180
E/flutter (11660): #4 get_yamnet_features_single_sample
yamnet_features.dart:38
E/flutter (11660):

It all looks like its just the way im setting up outputs, but im still not super experienced with interpretors and models in flutter, so i would appreciate some guidance or help.