- HealthMate(Chatbot): Given user input queries, generate output text addressing the queries powered by LLAMA model (Large Language Model).
- Multimodal Analyser: Other than text, if you want to give Image or Audio, the these standalone CNN model can be utilized. For instance if you want to check whether you have some skin lesion disease, you can simply capture a photo of your skin, and the CNN model classifies the skin lesion and the output is provided to the llama model which will assist you further.
- ONNX Export: Finally, the whole system is exported to ONNX to run on mobile devices.
UBUNTU="18.04"
CUDA="11.0"
CUDNN="8"
conda env create -f environment.yaml
conda activate qhack
Please download the data please use this link
../data/
ecg_data/
mit_train.csv
mit_test.csv
skin_data/
train/
ISIC_0034075.jpg
ISIC_0034075.jpg
train.csv
test/
test.csv
python medical_chatbot.py
# For skin_classification + chatbot
python unified_qhealthmate.py --input_image /path/to/your/image.jpg --image_model_path ./skin_cancer/models/best_model_final.pth --image_llama_vector_path ../data/skin_data/llama_embed_skin.npy
# For ecg_classification + chatbot
python unified_qhealthmate.py --input_audio /path/to/your/audio.wav --audio_model_path ./ecg/models/best_model_final.pth --audio_llama_vector_path ../data/ecg_data/llama_embed_ecg.npy
python skin_cancer/onnx_export.py --input_image path_to_input_image.jpg