hiranp / cookiecutter-python-cruft-poetry-tox-pre-commit-ci-cd

A Modern DevSecOps-centric Cookiecutter template for Python packages and/or projects

cookiecutter-cruft-poetry-tox-pre-commit-ci-cd

---

Documentation: https://cookiecutter-cruft-poetry-tox-pre-commit-ci-cd.readthedocs.io

Source Code: https://github.com/TeoZosa/cookiecutter-cruft-poetry-tox-pre-commit-ci-cd

Example Project: https://github.com/TeoZosa/cookiecutter-cruft-poetry-tox-pre-commit-ci-cd-instance

---

📝 Note
Originally inspired by @cjolowicz's fantastic cookiecutter-hypermodern-python project and eponymous Hypermodern Python article series.

A robust and extensible (nearly feature-complete), DevSecOps-centric Cookiecutter template for Python packages and/or projects with rich automated tooling for high-quality software development and maintenance, from testing and code quality to documentation and template synchronization.

Pre-configured to seamlessly run both locally and via the GitHub Actions CI/CD system to not only provide powerful guardrails and out-of-the-box audit trails, but significant quality of life improvements such as automated dependency upgrades, PR-based release notes, and many other high-level features for agile, defect-free software development.

Best of all, best practices are baked in order to provide you with the best foundation on which you can build your best work!

---

Table of Contents

• 🎉 Creating a New Project
• 🔗 Linking an Existing Project
• ✨ Features
  • 🚀 Project Standardization and Automation
    • 🔨 Developer Workflow Automation
    • 🌱 Conditionally Rendered Python Package or Project Boilerplate
  • ⚡ Performance
    • 📦 C-Extension Compilation
  • 🔧 Maintainability
    • 🏷️ Type Checking and Data Validation
    • ✅ Testing/Coverage
    • 🚨 Linting
    • 👷 CI/CD
  • 📉 Observability
    • 🔊 Logging
    • 🥅 Error Tracking
  • 🔒 Security
    • 🔏 Static Application Security Testing (SAST)
  • 📋 Accessibility
    • 📝 Project Documentation
    • ☑️ Design Documentation and Production Deployment Checklists
    • 🗃️ Architecture Documentation
• 🧑‍⚖️ Legal
  • 📄 License

🎉 Creating a New Project

Via cruft (recommended):

pip install --user cruft # Install `cruft` on your path for easy access
cruft create https://github.com/TeoZosa/cookiecutter-cruft-poetry-tox-pre-commit-ci-cd

Via cookiecutter:

pip install --user cookiecutter # Install `cookiecutter` on your path for easy access
cookiecutter gh:TeoZosa/cookiecutter-cruft-poetry-tox-pre-commit-ci-cd

Note: Cookiecutter uses gh: as short-hand for https://github.com/

🔗 Linking an Existing Project

If the project was originally installed via cookiecutter, you must first use cruft to link the project with the original template:

cruft link https://github.com/TeoZosa/cookiecutter-cruft-poetry-tox-pre-commit-ci-cd

Then/else:

cruft update

✨ Features

🚀 Project Standardization and Automation

🔨 Developer Workflow Automation

• Python packaging and dependency management with Poetry
• Project workflow orchestration with Make as an interface shim
  • Self-documenting Makefile; just type make on the command line to display auto-generated documentation on available targets:

self_documenting_makefile.mov

• Automated Cookiecutter template synchronization with cruft
• Test automation with Tox
• Code quality tooling automation and management with pre-commit
• Continuous integration and deployment with GitHub Actions

🌱 Conditionally Rendered Python Package or Project Boilerplate

• [Optional] Command-line interface boilerplate with Typer
• Project-specific Docker image Dockerfile¹ with production dependencies for a completely reproducible execution environment
• [Optional] Jupyter support¹

⚡ Performance

📦 C-Extension Compilation

• Python module to C-extension compilation (enabled by standard Python type hints) with Mypyc
  • Automatically configured for Python package builds (see the template's build.py file)

⚠️️ Warning
Mypyc is currently alpha software. It’s only recommended for production use cases with careful testing, and if you are willing to contribute fixes or to work around issues you will encounter.

🔧 Maintainability

🏷️ Type Checking and Data Validation

• Static type-checking with Mypy²
• Run-time type-checking with Typeguard
  • See the Typeguard user guide for usage overview

🔥 Tip
Complementary to type-checking, function-specific input validation is another useful technique. This helps eliminate an implicit knowledge violation from hidden argument requirements:

"Hidden argument requirements occur when a method signature implies a wider range of valid inputs than the method actually accepts. For example, accepting an int while only allowing numbers 1 to 10 is a hidden constraint." [1]

[1] C. Riccomini and D. Ryaboy, The Missing README: A Guide for the New Software Engineer, Paperback. No Starch Press, 2021.

✅ Testing/Coverage

• Testing with pytest
• Doctests with xdoctest²
• Performance testing with pytest-benchmark
• Property-based testing with Hypothesis
• Mutation testing with mutmut

ℹ️ Info
For a good overview of how property-based testing and mutation testing work together to improve the value and quality of your tests, see this stackoverflow post and the follow-up by the mutmut author.

• Code coverage with Coverage.py
• Coverage reporting with Codecov

🚨 Linting

• Code quality:
  • Ruff
    • A blazing-fast (10x-100x faster) replacement for Pylint, Flake8 (including major plugins), and more under a single, common interface
  • hadolint
  • Pylint²³
  • ShellCheck
• Code formatting:
  • Black²
  • isort²³
  • pyupgrade²³
  • shfmt
    • Configured by default to align with Google's Shell Style Guide
• General file formatting:
  • end-of-file-fixer
  • pretty-format-json
  • (trim) trailing-whitespace
• Unsanitary commits:
  • Secrets with detect-secrets
  • Debugger imports and py37+ breakpoint() calls with debug-statements
  • Large files with check-added-large-files
  • Invalid Python files with check-ast

👷 CI/CD

• Docker image builds and pushes to Docker Hub⁴
• Python package uploads to PyPI and TestPyPI¹⁴
• Documentation building and hosting with Read the Docs
• Release notes (PR-based) drafting and publishing with Release Drafter
• Dependency updates with Dependabot
  • Automated Dependabot PR merging with the Dependabot Auto Merge GitHub Action⁴
• Project issue labels management with GitHub Labeler
• Lightweight polyglot static analysis for code quality enforcement in addition to bug and security vulnerability identification with Semgrep

📉 Observability

🔊 Logging

• Structured logging with structlog-sentry-logger (via structlog)
  • Granular control flow context logging (via call stack introspection):
    • Namespaced module-specific loggers
    • Function name logging
  • Environment-dependent standard output stream log formatting:
    • Production: JSON logs
    • Development: Colorized human-readable logs, with JSON logs saved locally for retrospective analysis
  • [Optional] Exception logging to Sentry with structlog-sentry

🥅 Error Tracking

• [Optional] Exception monitoring with Sentry
  • see: the cookiecutter's .env file for a detailed activation guide

🔒 Security

🔏 Static Application Security Testing (SAST)

• Code vulnerabilities with Bandit²
• Python package dependencies vulnerabilities with Safety

📋 Accessibility

📝 Project Documentation

• Documentation building with Sphinx
  • Rich automatic documentation from type annotations and docstrings (NumPy, Google, etc.) with sphinx-autoapi
  • Automated emoji shortcode conversion⁵
• Docstring coverage with interrogate
• Automated README table of contents generation with markdown-toc
• Publishing to Confluence with Atlassian Confluence Builder for Sphinx

☑️ Design Documentation and Production Deployment Checklists

• Production service design documentation and deployment checklist templates with Mercari's production-readiness-checklist

🗃️ Architecture Documentation

• [Optional] Architecture knowledge management with Log4brains to systematically facilitate and record the planning process and context for each of a software system's architectural changes that occur over time and their consequences.
  • See: Documenting Architecture Decisions for an overview on Architecture Decision Records (ADR)

🧑‍⚖️ Legal

📄 License

cookiecutter-cruft-poetry-tox-pre-commit-ci-cd is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Footnotes

1. Conditionally rendered based on the configurations specified in the project setup phase to avoid tooling bloat ↩ ↩² ↩³

2. Jupyter notebook compatibility via nbQA ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

3. Via ruff ↩ ↩² ↩³

4. Requires definitions of one or more of the below repository secrets: AUTO_MERGE_DEPENDABOT_TOKEN DOCKERHUB_TOKEN DOCKERHUB_USERNAME PYPI_TOKEN TEST_PYPI_TOKEN ↩ ↩² ↩³

5. Custom setup steps in Sphinx configuration file for Read the Docs compatibility (see: conf.py:117-139) ↩