TTS onnxruntime bug in latest onnx-runtime (1.18)
l3utterfly opened this issue · comments
Tested with the example SherpaOnnxTTS android application.
To reproduce:
- Build android arm64_v8a (with onnx 1.18, using https://github.com/csukuangfj/onnxruntime-libs/releases/tag/v1.18.0)
- Copy built JNI into the example tts project
- Run app
- Type "hello, how are you"
- Click generate, once done speaking, click generate again
App crashes with the following error:
[E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running ScatterND node. Name:'/dp/flows.5/ScatterND_9' Status Message: updates tensor should have shape equal to indices.shape[:-1] + data.shape[indices.shape[-1]:]. updates shape: {0}, indices shape: {44,3}, data shape: {1,1,44}
From my tests, it seems the same tts instance cannot generate words/phonemes with the same text. For example, generating hello twice crashes as well. The only workaround is to release the tts instance and re-initialise it.
This error does not happen with onnx 1.17.x
Error log:
[E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running ScatterND node. Name:'/dp/flows.5/ScatterND_9' Status Message: updates tensor should have shape equal to indices.shape[:-1] + data.shape[indices.shape[-1]:]. updates shape: {0}, indices shape: {44,3}, data shape: {1,1,44}
2024-05-30 14:47:10.742 9707-9886 libc++abi com.k2fsa.sherpa.onnx E terminating due to uncaught exception of type Ort::Exception: Non-zero status code returned while running ScatterND node. Name:'/dp/flows.5/ScatterND_9' Status Message: updates tensor should have shape equal to indices.shape[:-1] + data.shape[indices.shape[-1]:]. updates shape: {0}, indices shape: {44,3}, data shape: {1,1,44}
2024-05-30 14:47:10.743 9707-9886 libc com.k2fsa.sherpa.onnx A Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 9886 (Thread-4), pid 9707 (fsa.sherpa.onnx)
2024-05-30 14:47:10.931 9889-9889 DEBUG crash_dump64 A Cmdline: com.k2fsa.sherpa.onnx
2024-05-30 14:47:10.931 9889-9889 DEBUG crash_dump64 A pid: 9707, tid: 9886, name: Thread-4 >>> com.k2fsa.sherpa.onnx <<<
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #01 pc 00000000000c5620 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (abort_message+260) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #02 pc 00000000000c4f14 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (demangling_terminate_handler()+268) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #03 pc 00000000000c4dd0 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (std::__terminate(void (*)())+12) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #04 pc 00000000000de0f0 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (__cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*)+32) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #05 pc 00000000000de064 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (__cxa_throw+128) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #06 pc 00000000000acb70 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (Ort::detail::ThrowStatus(Ort::Status const&)+292) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #07 pc 00000000000e3018 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (Ort::detail::SessionImpl<OrtSession>::Run(Ort::RunOptions const&, char const* const*, Ort::Value const*, unsigned long, char const* const*, unsigned long)+220) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #08 pc 00000000001b0af4 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsModel::Impl::RunVitsPiperOrCoqui(Ort::Value, long, float)+1256) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #09 pc 00000000001b055c /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsModel::Impl::Run(Ort::Value, long, float)+104) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #10 pc 00000000001b0478 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsModel::Run(Ort::Value, long, float)+48) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #11 pc 00000000001ac36c /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsImpl::Process(std::__ndk1::vector<std::__ndk1::vector<long, std::__ndk1::allocator<long> >, std::__ndk1::allocator<std::__ndk1::vector<long, std::__ndk1::allocator<long> > > > const&, int, float) const+536) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #12 pc 00000000001a39b8 /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsImpl::Generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, long, float, std::__ndk1::function<void (float const*, int, float)>) const+1492) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #13 pc 00000000001b42ac /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTts::Generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, long, float, std::__ndk1::function<void (float const*, int, float)>) const+172) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #14 pc 00000000000199ec /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-jni.so (Java_com_k2fsa_sherpa_onnx_OfflineTts_generateWithCallbackImpl+376) (BuildId: da6dfa3ce2c1ea48c2425e7075bc51b5e0bbf067)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #21 pc 00000000000033f4 [anon:dalvik-classes3.dex extracted in memory from /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.OfflineTts.generateWithCallback+0)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #26 pc 0000000000001f90 [anon:dalvik-classes3.dex extracted in memory from /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity.onClickGenerate$lambda$3+0)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #31 pc 0000000000001a78 [anon:dalvik-classes3.dex extracted in memory from /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity.$r8$lambda$GXwRBIBrY-N067P2cW25xsk89bs+0)
2024-05-30 14:47:10.932 9889-9889 DEBUG crash_dump64 A #36 pc 00000000000018c4 [anon:dalvik-classes3.dex extracted in memory from /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity$$ExternalSyntheticLambda1.run+0)
Please use 1.17.1 instead.
Some ASR models also have issues with onnxruntime > 1.17.1, including 1.18.0
You can see we are still using onnxruntime 1.17.1