P_WE_ML_docker: creating images of semantic models, preprocessing methods and deployment with flask
pipreqs
- Generate requirements.txt file for any project based on imports
pip install pipreqs
pipreqs /home/project/location
Successfully saved requirements file in /home/project/location/requirements.txt
nltk==3.4.5
pandas==1.0.1
Flask==1.1.1
scikit_learn==0.23.2
numpy==1.13.3
(PS: if the requirements.txt exits, you have to delete it before) learn more
Software |
---|
flask, scikit-learn, docker, MQTT, pandas, numpy, nltk |
virtualenv --python=python3 env
source env/bin/activate
pip install -r requirements.txt
cd
mkdir docker_project
cd docker_project
git init
git clone https://github.com/adlaneKadri/P_WE_ML_docker.git
cd P_WE_ML_docker
you find models folder here, and download it uzip models folder
unzip P_WE_ML/models.zip
P_WE_ML_docker
├── P_WE_ML
│ └── frontend
| └── backend
| └── dataset
| └── ml_hub
| └── decision_tree/
| └── gradient_boosting/
| └── mlp/
| └── random_forest/
| └── preprocessing
| └── processing.py
| └── word_embeding
| └── bow/
| └── tfidf/
| └── utils_
| └── config.py
| └── models
| └── logs
├── docker
├── docker-compse
├── P_WE_ML_docker
└── Vagrantfile
to run the frontend
cd frontend
python3 display_flask.py
ON your browser :
http://127.0.0.1:5000/
to run the preprocessing
cd backend/preprocessing
python3 processing.py
to run the word embeding
cd backend/bow
python3 bag_of_word.py
to run a machine learning model
cd backend/ml_hub
>choose your model (for example: decision tree)
python3 decision_tre.py
PS --> volumes: use absolute path (-v all path of your folder)
Web application
cd frontend
sudo docker build -t web:1.0.0 .
sudo docker run -v /mypath_to_docker_project/P_WE_ML_docker/docker/models:/app/models -v /mypath_to_docker_project/P_WE_ML_docker/docker/utils_:/app/utils_ -p 5000:5000 web:1.0.0
ON your browser :
http://0.0.0.0:5000/
Data preprocessing
cd backend/preprocessing
sudo docker build -t processing:v1 .
sudo docker run -v /mypath_to_docker_project/P_WE_ML_docker/docker/backend/dataset:/app/dataset -v /mypath_to_docker_project/P_WE_ML_docker/docker/utils_:/app/utils_ processing:v1
Word embeding (for example tfidf)
cd backend/tfidf
sudo docker build -t tfidf:v1 .
sudo docker run -v /mypath_to_docker_project/P_WE_ML_docker/docker/backend/dataset:/app/dataset -v /mypath_to_docker_project/P_WE_ML_docker/docker/models:/app/models -v /mypath_to_docker_project/P_WE_ML_docker/docker/utils_:/app/utils_ tfidf:v1
Machine learning model (for example decision tree)
cd backend/ml_hub/decision_tree
sudo docker build -t decision_tree:v1 .
sudo docker run -v /mypath_to_docker_project/P_WE_ML_docker/docker/backend/dataset:/app/dataset -v /mypath_to_docker_project/P_WE_ML_docker/docker/models:/app/models -v /mypath_to_docker_project/P_WE_ML_docker/docker/utils_:/app/utils_ decision_tree:v1
- please create 2 folders (logs, models)
mkdir /mypath_to_docker_project/P_WE_ML_docker/docker/logs
mkdir /mypath_to_docker_project/P_WE_ML_docker/docker/models
Web application
cd frontend
sudo docker build -t web:1.0.0 .
sudo docker run -v /mypath_to_docker_project/P_WE_ML_docker/docker/models:/app/models -v /mypath_to_docker_project/P_WE_ML_docker/docker/utils_:/app/utils_ -v /mypath_to_docker_project/logs:/app/logs -p 5000:5000 web:1.0.0
ON your browser :
http://0.0.0.0:5000/
Data preprocessing
cd backend/preprocessing
sudo docker build -t processing:v1 .
sudo docker run -v /mypath_to_docker_project/backend/dataset:/app/dataset -v /mypath_to_docker_project/utils_:/app/utils_ -v /mypath_to_docker_project/logs:/app/logs processing:v1
Word embeding (for example tfidf)
cd backend/tfidf
sudo docker build -t tfidf:v1 .
sudo docker run -v /mypath_to_docker_project/backend/dataset:/app/dataset -v /mypath_to_docker_project/models:/app/models -v /mypath_to_docker_project/utils_:/app/utils_ -v /mypath_to_docker_project/logs:/app/logs tfidf:v1
Machine learning model (for example logistic Regrission)
cd backend/ml_hub/logistic_regrission
sudo docker build -t logisticRegrission:v1 .
sudo docker run -v /mypath_to_docker_project/backend/dataset:/app/dataset -v /mypath_to_docker_project/models:/app/models -v /mypath_to_docker_project/utils_:/app/utils_ -v /mypath_to_docker_project/logs:/app/logs logisticRegrission:v1
To run vagrant (we are using ubuntu20 version in Vagrantfile configuration)
vagrant up
To destroy the current machine with all params
vagrant destroy
To access in VM with ssh
vagrant ssh