NVIDIA / trt-llm-rag-windows

A developer reference project for creating Retrieval Augmented Generation (RAG) chatbots on Windows using TensorRT-LLM

NVIDIA/trt-llm-rag-windows Issues

Update to Llama 3 8B model
Updated 7 days ago1
ModuleNotFoundError: No module named 'tensorrt_llm'
Closed 14 days ago4
Not enough free memory to build TRT engine
Closed 14 days ago3
Missing config/preferences.json and Cookie Handling Errors
Closed 15 days ago1
Unable to stream results. TypeError: 'NoneType' object is not iterable
Updated 20 days ago1
Fine-tune WhereIsAI/UAE-Large-V1 embeddings for Models First ?
Closed 20 days ago1
I am building a software like this right now, and finding this is depressing
Closed 20 days ago6
Support for Markdown files
Updated 20 days ago7
Installed both models in Chat-With-RTX but can only use Mistral 7B int4
Updated 20 days ago5
invalid session error
Closed 20 days ago4
nrw
Closed 20 days ago
Can Chat with RTX support Pre-trained models as dataset input?
Updated 20 days ago2
Can i install without 8gbvram?
Closed 20 days ago1
Can it run on dual GPU ?
Closed 20 days ago2
feat: can it read repository
Closed 20 days ago1
Large amount of files get stuck before loading
Updated 20 days ago1
Connection error
Closed 20 days ago4
Not able to generate engine on RTX4060 Laptop 8GB 100% CPU being utilised
Updated 20 days ago2
How to switch language?!
Closed 20 days ago3
Dtype value read by app.py empty. Gets error message : Unsupported dtype
Updated 20 days ago2
ModuleNotFoundError: No module named 'tensorrt'
Closed 20 days ago15
AttributeError: 'WeightOnlyGroupwiseQuantLinear' object has no attribute 'prequant_scaling_factor'
Updated 20 days ago2
Incorrect instructions to build TRT engine
Closed 20 days ago8
LInux
Closed 20 days ago10
CHM file support
Updated 20 days ago1
How to implement chat history memory and system prompt?
Updated a month ago1
ChatWithRTX Installer is corrupt.
Closed a month ago3
[TensorRT-LLM][ERROR] CUDA runtime error in cudaDeviceGetDefaultMemPool
Closed a month ago2
not able to debug this even with gdb
Updated a month ago1
default response language
Closed a month ago2
What version of Cuda is needed?
Closed a month ago5
Override stringent GPU requirements
Updated 2 months ago1
build TRT engine fail
Closed 3 months ago6
garbage output ?
Closed 3 months ago1
pycryotodome
Closed 3 months ago1
Enhancement: Support for reading PDFs with partial DRM (AES) - include PyCryptodome dependency
Updated 3 months ago4
ModuleNotFoundError: No module named 'tensorrt_llm.bindings'
Closed 4 months ago5
Some questions crash the app
Closed 4 months ago2
I had to install Microsoft MPI to be able to run it otherwise it was throwing error
Closed 4 months ago1
Building TRT Engine on Windows 11 Results in LoraConfig.from_hf() Arguments Error
Updated 4 months ago3
Incredibly unclear instructions
Updated 4 months ago3
Minimum Requirements GPU RAM
Closed 4 months ago1
ModuleNotFoundError: No module named 'llama_index'
Closed 4 months ago8
Error Code 1: Serialization (Serialization assertion stdVersionRead == kSERIALIZATION_VERSION failed.Version tag does not match.
Updated 4 months ago5