neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs

https://neuralmagic.com/deepsparse/

neuralmagic/deepsparse Issues

yolo-v8 in onnx-runtime outperforms deepsparse on iMX8
Closed 3 months ago6
Failed to import dynamically quantized roberta model in ONNX format.
Updated 24 days ago1
Conversion and display of yolov8 rtsp video inference results
Updated a month ago2
Unable to load DeepSparseSentenceTransformer
Updated a month ago2
How can I use deepsparse instead of ultralytics
Closed 2 months ago2
Unknown Pipeline task yolov8. Currently supported tasks are ['text_generation', 'opt', 'llama', 'code_gen', 'code_generation', 'codegen', 'image_classification', 'mpt']
Closed 2 months ago2
YOLOv8 - Display bounding boxes and classes names in image using python.
Closed 3 months ago3
How can I make proper request to server
Updated 2 months ago1
[Question] about converting onnx model with dynamic batch size input to deepsparse model
Updated 2 months ago2
Using output_value as "token_embeddings" is broken for Sentence Transformer
Closed 3 months ago2
Paper: New algorithm for pruning
Closed 3 months ago3
Assertion at src/lib/core/topology.cpp:627
Closed 3 months ago1
How to use for the fintuned roberta model for text classification
Updated 3 months ago3
deepsparse.TextGeneration doesn't accept `trust_remote_code` as an arg anymore
Updated 3 months ago1
Research: 4-bit quantization
Closed 3 months ago5
Purpose of exporter.export_onnx(sample_batch=torch.randn(1, 1, 28, 28))
Closed 3 months ago4
docker access denied error
Closed 3 months ago8
Unsupported ONNX type 10 for FP16
Closed 3 months ago5
Python3.12?
Closed 4 months ago3
Support for Raspberry Pi 4
Closed a year ago2
Segmentation fault (core dumped)
Closed 7 months ago6
transformers_embedding-extraction for text-generation tasks
Closed 7 months ago3
NM: error: Node (/model/Add_1) Op (Add) [ShapeInferenceError] Incompatible dimension
Closed 8 months ago5
object detection on video with deepsparse
Closed 8 months ago4
Question on quantization size
Closed 8 months ago1
Assertion `!cache_sizes.empty()' failed
Closed 8 months ago2
Does he support stable diffusion or fooocus, or other image generation acceleration?
Closed 8 months ago1
Adding Contributors Section to readme.md
Closed 9 months ago1
https://www.therapyinsightspractice.com
Closed 9 months ago1
Support for Llama2?
Closed 9 months ago2
How to use Benchmark feature on custom dataset
Closed 10 months ago1
Can't use DeepSparse with VITS ONNX file from Coqui TTS.
Closed 10 months ago16
Encountered an issue when trying to optimize Donut model.
Closed 10 months ago7
DeepSparse uses 100% of the CPU.
Closed a year ago2
Error when compiling YOLO-NAS model
Closed a year ago11
Deepsparse in colab
Closed a year ago4
I see that SparseGPT has been integrated into your project. May I know which file the specific modifications are in? How to use it specifically?
Closed a year ago3
Support numpy > 1.21.6
Closed a year ago3
conflict error
Closed a year ago2
Build instructions for other CPU architectures
Closed a year ago1
Slow inference time in yolov5 speed<12 fps
Closed a year ago5
Output is not the same as ONNX
Closed a year ago2
Please install deepsparse[yolo] to use this pathway
Closed a year ago1
Neural Magic: Encountered exception while trying to read arch.bin: Unable to determine instruction set
Closed a year ago4
number of cores used in pipeline
Closed a year ago4
pruned/quantized yolov8 model is slower
Closed a year ago2
KeyError: 'length'
Closed a year ago2
Misleading docs for deepsparse.transformers.eval_downstream
Closed a year ago
When can support Windows？
Closed a year ago1
Cv2. Resize error yolov8 task in deepspatse. server onnx model
Closed a year ago5