ERROR: "The parameter is incorrect." when running phi3 DML C example on Windows 10
MajorPainTheCactus opened this issue · comments
Hi so I've just built the latest version of onnxruntime-genai c example which created a phi3.exe by VS2022 that using cmake 3.26.3 excludes CUDA and uses DirectML. When I run this with various phi 3 models downloaded from hugging face:
https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/tree/main (directml folder)
Debug command line: C:\Development\Phi-3-mini-4k-instruct-onnx\directml\directml-int4-awq-block-128
https://huggingface.co/microsoft/Phi-3-medium-4k-instruct-onnx-directml/tree/main
Debug command line: C:\Development\Phi-3-medium-128k-instruct-onnx-directml\directml-int4-awq-block-128
I get a similar error:
Text Debug Output in VS2022:
Exception thrown at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: _com_error at memory location 0x0000003C32EF34C0.
C:\__w\1\s\Product\ComputeShaderFactory.cpp(398)\DirectML.dll!00007FFF57CE26E6: (caller: 00007FFF57E00FC5) Exception(1) tid(a48c) 80070057 The parameter is incorrect.
Exception thrown at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: wil::ResultException at memory location 0x0000003C32EF2E60.
Exception thrown at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000.
C:\__w\1\s\Product\DmlDevice.cpp(238)\DirectML.dll!00007FFF57F2D23B: (caller: 00007FFF57CBC136) ReturnHr(1) tid(a48c) 80070057 The parameter is incorrect.
Msg:[C:\__w\1\s\Product\ComputeShaderFactory.cpp(398)\DirectML.dll!00007FFF57CE26E6: (caller: 00007FFF57E00FC5) Exception(1) tid(a48c) 80070057 The parameter is incorrect.
]
C:\__w\1\s\SharedToolingLib/External/Include/OperatorCache.h(86)\DirectML.dll!00007FFF57CBC1BC: (caller: 00007FFF57EA9236) Exception(2) tid(a48c) 80070057 The parameter is incorrect.
Exception thrown at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: wil::ResultException at memory location 0x0000003C32EF87E0.
Exception thrown at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: [rethrow] at memory location 0x0000000000000000.
C:\__w\1\s\Product\DmlDevice.cpp(796)\DirectML.dll!00007FFF57F2D1EE: (caller: 00007FFF59FA66D9) ReturnHr(2) tid(a48c) 80070057 The parameter is incorrect.
Msg:[C:\__w\1\s\SharedToolingLib/External/Include/OperatorCache.h(86)\DirectML.dll!00007FFF57CBC1BC: (caller: 00007FFF57EA9236) Exception(2) tid(a48c) 80070057 The parameter is incorrect.
]
D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusionHelper.cpp(576)\onnxruntime.dll!00007FFF59FA6D86: (caller: 00007FFF5A038E7D) Exception(1) tid(a48c) 80070057 The parameter is incorrect.
Exception thrown at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: wil::ResultException at memory location 0x0000003C32EFA6B0.
Exception thrown at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: Ort::Exception at memory location 0x0000003C32EFEE30.
Exception thrown at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: std::runtime_error at memory location 0x0000003C32EFF6A8.
Unhandled exception at 0x00007FF820A2BA99 in phi3.exe: Microsoft C++ exception: std::runtime_error at memory location 0x0000003C32EFF6A8.
With the executable outputting this:
Screenshot:
Text Output:
-------------
Hello, Phi-3!
-------------
C++ API
Creating model...
Creating tokenizer...
Prompt:
def is_prime(num):
2024-05-25 21:54:45.7917787 [E:onnxruntime:onnxruntime-genai, sequential_executor.cc:516 onnxruntime::ExecuteKernel] Non-zero status code returned while running DmlFusedNode_0_0 node. Name:'DmlFusedNode_0_0' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusionHelper.cpp(576)\onnxruntime.dll!00007FFF59FA6D86: (caller: 00007FFF5A038E7D) Exception(1) tid(a48c) 80070057 The parameter is incorrect.
C:\Development\onnxruntime-genai\examples\c\bin\Debug\phi3.exe (process 13936) exited with code -1.
Any idea what I might be doing wrong?
Many thanks!
My exe directory:
DebugDir.zip
cmake configured options:
@kunal-vaishnavi do let me know if you need any help reproducing. I can provide memory dumps etc. I want to distribute Phi3 with my application so this is very important to me.
Can you try building both ONNX Runtime and ONNX Runtime GenAI from source and running the example again to see if the error goes away? There have been multiple DirectML fixes that have been merged into both recently.
MajorPainTheCactus I just tried running the example on my end and didn't see any problem running with DML.
I updated the example script to make it simpler to use with the phi3 language model in this PR: #607. Could you please give it a try?
-------------
Hello, Phi-3!
-------------
C API
Creating model...
Creating tokenizer...
Prompt:
Tell me a joke on Pirates.
Generating response...
Why don't pirates like to go to the doctor?
Because they always fear the "medical" bills!
(Note: While this joke is light-hearted, it's essential to remember that jokes about pirates or any other group should be respectful and not perpetuate stereotypes.)
Also, could you try the command line following the instructions here: https://github.com/microsoft/onnxruntime-genai/blob/51688ae75f7060d2db52e48ddd6970b1ee9d0d13/examples/c/README.md#build-the-cmake-project instead of cmake GUI?
@baijumeswani Thank you ever so much for looking into this. I'll update, follow those instructions and see where it gets me and let you know. Thanks again!