ML Development Guide

This is a general guide about package installation and system setup

Windows System Setup

Use miniconda to manage virtual environments for projects
Check your system's configuration (ie, x86, 64 bit or ARM)
Download the appropriate installer from https://docs.anaconda.com/free/miniconda/
Run the installer and install miniconda at your preffered path
Create new environment with latest python: conda create -n llm python
Activate the new environment: conda activate llm

You can potentially need this in the future while developing ML applications and installing packages
To prevent future errors like this one: https://github.com/Akshay-Dongare/Error-Cpp-Build-Tools
Install Build Tools using these steps:

Download the Visual Studio Build Tools installer at https://visualstudio.microsoft.com/visual-cpp-build-tools/
Install the 'Desktop Development with C++' components using Visual Studio Build tool. (approx 6.84GB total)
Locate the directory where MSBuild is installed (typically "C:\Program Files (x86)\Microsoft Visual Studio\20xx\BuildTools\MSBuild\Current\Bin" where the xx in 20xx signifies the version of Microsoft Build Tools you have downloaded) and add it to the PATH variable (This is an extremely important step as it ensures that Build Tools can be found by other programs like pip). Add it to the System Environment PATH variable rather than User Environment PATH variable.
Install cmake (pip install cmake) if not already installed.
RESTART your computer.

Use Docker Volumes (docker run -v) to map current working dir to container dir to get real time updates in the container
Install VS code extensions: Dev Containers (by Microsoft), Docker, Remote Development (an extension pack by Microsoft)
Then from vs code, connect remote to running docker container
In the new window in Remote VS code, install python extension to get all the vs code features like autocomplete, refactor and peek defination
docker-compose.yml file allows to specify services and to run our container using a single simple command: docker-compose up
If you change something in requirements.txt, for example you added a python module dependancy for the new service you added in docker-compose.yml so that you can use that service in your python code; you must build the docker image again like so: docker-compose up --build -d (d is for detached mode)
To setup debugging, add debugpy module to python dependancies requirements.txt, then in docker-compose.yml file, under ports: map the 5678 port to 5678 (on local machine). Now in the Remote Docker container attached VS Code window, go to debugger and select attach to remote debug server, now enter localhot and port 5678
Source: https://www.youtube.com/watch?v=6OxqiEeCvMI&t=108s&ab_channel=Docker