microsoft / onnxruntime-genai

Generative AI extensions for onnxruntime

microsoft/onnxruntime-genai Issues

Cannot get device_type from Python API
Updated 3 days ago
Usage example for `get_output()`
Updated 3 days ago1
[Documentation Request] pip install onnxruntime-genai --pre command does not work
Updated 3 days ago14
Memory leak during back-to-back inferences
Updated 3 days ago
Device API throws exception
Updated 3 days ago3
Using onnxruntime-genai-directml without CUDA
Updated 4 days ago3
AttributeError due to a lack of default value in phi3-qa.py
Closed 4 days ago3
[Feature Request] phi-3-small-128k-onnx-cpu model
Updated 4 days ago3
segmentation fault when running long sequence input to Phi3-128k-instruct-onnx
Closed 4 days ago5
How to implement support for concurrency?
Updated 4 days ago2
[Mobile] System.DllNotFoundException "onnxruntime-genai" when debugging on Android
Updated 4 days ago3
[Mobile] System.DllNotFoundException "onnxruntime-genai" when debugging on Android
Closed 4 days ago3
Errors when running Phi-3 on NVIDIA CUDA with `onnxruntime_genai`
Updated 4 days ago10
ONNXRuntime genai fine tuning
Closed 4 days ago6
No support for multiple image while using Phi-3-vision (Inference)
Closed 4 days ago1
onnx for phi-3 mini
Updated 4 days ago6
ERROR: "The parameter is incorrect." when running phi3 DML C example on Windows 10
Updated 4 days ago1
Phi-3-Mini fails to execute on long prompts on Intel integrated GPU with DirectML
Updated 5 days ago7
Phi3-Vision ONNX models
Updated 6 days ago2
[.NET] Genny chat-bot sample doesn't support DirectML and Phi-3
Updated 6 days ago2
Questions about the `temperature` config in search options and its unexpected/incorrect behavior
Closed 6 days ago5
ONNXRuntime-genai doesn't release GPU memory after first inference
Updated 6 days ago
Phi3 Vision models feedback and questions
Updated 6 days ago4
cant install on aarch64 Raspbian64 Bullseye
Closed 7 days ago6
With Latest GeForce Driver v555 - Microsoft.ML.OnnxRuntimeGenAI.OnnxRuntimeGenAIException: The GPU will not respond to more commands, most likely because of an invalid command passed by the calling application.
Updated 7 days ago12
pip install --pre onnxruntime-genai
Updated 7 days ago4
Can this library be build without having to link OpenCV?
Updated 7 days ago4
Phi-3-mini-4k-instruct-onnx model generates nonsensical results when prompt is longer than half of the context window length
Updated 7 days ago2
DirectML error when input to the `Phi-3-mini-128k-instruct-onnx` model is longer than 2048 tokens
Updated 7 days ago2
Does onnxruntime or onnxruntime_genai support tool_calling?
Closed 7 days ago2
Loading Image from bytes
Updated 7 days ago
DirectML error when input is longer than half of the maximum context length
Updated 8 days ago
Ort::InitApi() is only called after Generator related code is invoked
Updated 9 days ago
Can I use onnxruntime-genai with a grammar for output like llama.cpp?
Closed 9 days ago2
Unknown value: inputs_embeds
Updated 9 days ago9
Is OgaGenerator a one-time object?
Closed 9 days ago2
AttributeError: module 'onnxruntime_genai' has no attribute 'Model'
Updated 9 days ago2
Comparison of multiple images with the Phi-3 Vision model
Closed 9 days ago1
cannot install onnxruntime-genai on CPU based, Intel Mac
Closed 9 days ago4
AttributeError on processor & create_multimodal_processor.
Closed 10 days ago1
Looking for onnx version of microsoft/Phi-3-vision-128k-instruct
Closed 14 days ago5
[.NET] Phi-3 sample response is the same as input prompt
Closed 13 days ago2
RuntimeError: Unsupported model_type in config.json: phi3small
Closed 14 days ago4
"Not enough memory resources are available to complete this operation" error when running `Phi-3-mini-128k-instruct-onnx` on DirectML without explicitly specifying 'max_length' in search options
Closed 14 days ago2
Inspect GeneratorParams (and possible other class) in C# & Java
Updated 14 days ago
Support Llama / Hugging Face's Universal Format (GGUF)
Updated 15 days ago
System.DllNotFoundException: 'Unable to load DLL 'onnxruntime-genai' or one of its dependencies (Visual Studio 2022)
Closed 15 days ago7
TokenizerStream returns incomplete strings
Closed 15 days ago7
Benchmarking Phi-3 on CPU
Closed 20 days ago8
onnxruntime_genai.onnxruntime_genai.OrtException: Non-zero status code returned while running GroupQueryAttention node. Name:'/model/layers.0/attn/GroupQueryAttention' Status Message: cos_cache dimension 1 must be <= head_size / 2 and a multiple of 8.
Closed 23 days ago2