Install Conda

I recommend installing Conda Miniforge to handle virtual environments.

cd ~/Downloads

wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-x86_64.sh

bash Miniforge3-MacOSX-x86_64.sh

Create a virtual enviroment and activate it

Conda environments are isolated environments where packages and dependencies can be installed and managed separately from the system-wide packages. This allows for different versions of packages and dependencies to be used in different environments, without interfering with each other.

conda create --name py10 python=3.10

conda activate py10

Pre-commit

A framework for managing and maintaining multi-language pre-commit hooks.

Instalation

conda install pre-commit

pre-commit install

pre-commit hooks

Git hook scripts are useful for identifying simple issues before submission to code review.

check-added-large-files
makes sure no large, probably data files, are added to git
check-json
check-toml
check-yaml
checks for a valid format of these three configuration files
requirements-txt-fixer
orders the requirements alphabetically
trailing-whitespace
removes white spaces after the end of the line
id: end-of-file-fixer
adds an empty line to the end of the file
id: no-commit-to-branch
args: [--branch, master, --branch, main]

isort

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.

profile = "black"
makes sure that inplace changes compatible with black
line_length = 88
allows up to 88 characters per line

black

Black is a PEP 8 compliant opinionated formatter with its own style.

line-length = 88
allows up to 88 characters per line

flake8

A Tool For Style Guide Enforcement

docstring-convention = numpy
max-line-length = 88
select = B, C, E, F, W, B9, ISC
ignore = E203, E402, E501, E722, B001, W503

mypy

Mypy is a static type checker for Python.

ignore_missing_imports = true
no_implicit_optional = true
check_untyped_defs = true
strict_equality = true
warn_redundant_casts = true
warn_unused_ignores = true
show_error_codes = true
disallow_any_generics = true
disallow_incomplete_defs = true
disallow_untyped_defs = true

bandit

Bandit is a tool designed to find common security issues in Python code.

DVC

DVC allows you to track, manage, and version your data and models in a similar way to how Git tracks code.

Installation

conda install dvc

Initialization

dvc init
dvc remote add <remote-path>

add files or folder to dvc

dvc add dvc/<local-path>

push files to remote server

dvc push

GitHub Actions

A continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline.

The GitHub Actions workflow located at github/workflows/ci.yaml is triggered on pull requests.

It contains two jobs, "Lint" and "Test", that run on the latest version of Ubuntu.

The "Lint" job sets up Python 3.10, and then installs the dependencies needed for linting. It then runs various linters such as isort, black, flake8, mypy and bandit to check for code formatting, style and security issues.

The "Test" job also sets up Python 3.10, installs dependencies, including dev dependencies, and runs pytest to execute the test cases.

This GitHub CI workflow is used for checking the code quality, formatting and security before merging the pull request.

drsantos89 / template-ds-ds