Python Templates

Python template scripts I've collated over time, as described below.

Folder	Description
`1_generic_templates`	Generic template python scripts
`2_data_related_templates`	Data-related python scripts, e.g. containing standardised code for DB/API calls
`3_other_templates`	Collection of templates that aren't generic or data-related!

Python Style Guide

Throughout the template scripts developed, the following style guide of do’s and don’ts for Python development has been used, covering the following main areas:

Naming Conventions
Code Layout
String Formatting: Use f-strings
Function and Method Definitions
Comments and Docstrings
Error Handling
General Guidelines

ℹ️ Summary: Use Python Linters and Automatters to Facilitate Coding Standards

- Implementing the below styles/coding standards requires proactive efforts from end users.
- I recommend teams use linters and code autoformatters to maintain code quality and standardised formatting.
- For more information, see <TODO - link here>

1. Naming Conventions

1.1. Case Styles, Per-Python Object

In summary:

Use snake_case for all module, package, function, and variable names.
Use CamelCase for classes.
Use SCREAMING_SNAKE_CASE for constants.

What is snake_case? Expand the menu below for more details.

Expand for details

snake_case combines words by replacing each space with an underscore (_), and all letters are lowercase, as follows:

Raw: user login count

Snake case: user_login_count

The following link explains the differences between different case styles.
One of the benefits of snake_case is that many of the allowed characters are compatible across S3 and Snowflake.

Case Style Specification

Expand for details

Related to	Naming Convention	Example
1. Module names	`snake_case` (i.e., `lowercase_with_underscores`)	`lowercase_with_underscores.py`
2. Package names	`snake_case`	`lowercase_with_underscores`
3. Class names	`CamelCase`	`ExampleClass`
4. Function and method names	`snake_case`	`def example_function():`
5. Variable names	`snake_case`	`working_dir = os.getcwd()`
6. Constants	`SCREAMING_SNAKE_CASE` (i.e., `UPPERCASE_WITH_UNDERSCORES`)	`PI = 3.14`

1.2. Avoid Abbreviations

Function names, variables, and filenames should all be meaningful, descriptive, and eschew abbreviations.
For example (click to expand)
Good:
```
def load_customer_data():
```
Bad:
```
def load_data():
```
Don't use abbreviations that are ambiguous to readers outside your projects.
Don't abbreviate by deleting letters within a word.

2. Code Layout

2.1. General

PEP 8 is the official style guide for Python code. It provides a set of conventions for writing readable and consistent code. As such, we should use its main high-level code layout recommendations, listed below:

Using four spaces for indentation
Limiting lines to 79 characters
Using blank lines to separate functions and classes

Expand the menu below for further details on these (including examples).

Example (click to expand)

Indentation

Description	Example
Use four spaces for indentation (i.e., avoid tabs.)	N/A

Line length

Description	Example
Limit lines to a maximum of 79 characters.	N/A

Whitespace: imports, classes, and functions

Description	Example
Use a newline after imports and before the class or function definition.	Expand menu below

Example (click to expand)

Good

import os
...

from requests import HTTPError


def example_function():

Bad

import os
...

from requests import HTTPError
def example_function():

Whitespace: methods and classes

Description	Example
Separate methods and classes with two blank lines.	Expand menu below

Example (click to expand)

Good

import os
...

from requests import HTTPError


def example_function():

Bad

import os
...

from requests import HTTPError
def example_function():

Whitespace: operators and commas

Description	Example
Use a single space around operators and after commas.	Expand menu below

Example (click to expand)

Good

return sharepoint.get_site(host, 'sites/' + site_name.replace(" ", ""))

Bad

return sharepoint.get_site(host,'sites/'+site_name.replace(" ", ""))

Whitespace: code readability

Description	Example
Code MUST be spaced logically to maintain readability.	Expand menu below

Example (click to expand)

Good

# Pipeline parameters
src_bucket = self.s3_bucket_src
logging.info(f”source_bucket = {src_bucket}”)

target_bucket = self.s3_bucket_target
logging.info(f”target_bucket = {target_bucket}”)

Bad

# Pipeline parameters
src_bucket = self.s3_bucket_src
logging.info(f”source_bucket = {src_bucket}”)
target_bucket = self.s3_bucket_target
logging.info(f”target_bucket = {target_bucket}”)

2.2. Imports

Import modules on separate lines.
Use absolute imports whenever possible.
Avoid wildcard imports (from module import *).
Place imports in the following order: standard library modules, third-party modules, and local modules.

For further details (with examples), expand the menu below.

Click to expand

Description	Example
Import modules on separate lines.	Expand menu below

Click to expand

Good

import os
import re

Bad

import os, re

Description	Example
Use absolute imports whenever possible.	Expand menu below

Click to expand

Good

from package2.subpackage1.module5 import function2

Bad

from .subpackage1.module5 import function2

Description	Example
Avoid wildcard imports (`from module import *`)	Expand menu below

Click to expand

Good

from math import ceil

Bad

from math import *

Description	Example
Place imports in the following order: 1. standard library modules, 2. third-party modules, 3. and local modules.	Expand menu below

Click to expand

Good

import os # standard modules
import boto3 # third-party modules
import my_local_module # local modules

Bad

import boto3 # third-party modules
import my_local_module # local modules
import os # standard modules

3. String Formatting: Use `f-strings`

f-strings should be used where possible. They provide a clean way to format strings.
Avoid using str().format.
Detailed examples of f-strings in action can be found here.

4. Function and Method Definitions

Use type declarations for method arguments and return values.

Click to expand

Good

def get_secret_by_name(secret_name: str) -> str:

...

secret = <code here>

return secret

Bad

def get_secret_by_name(secret_name):

...

secret = <code here>

return s

Place default arguments at the end of the argument list.

Click to expand

Good

def student(firstname, lastname ='Mark', standard ='Fifth'):

Bad

def student(lastname ='Mark', standard ='Fifth', firstname):

Define function parameters without spaces around the equals sign.

Click to expand

Good

func(1, 2, axis='x', angle=90, size=450, name='foo bar')

Bad

func(1, 2, axis = 'x', angle = 90, size = 450, name = 'foo bar')

Use a single space after the comma in function definitions.

Click to expand

Good

func(1, 2, axis='x', angle=90, size=450, name='foo bar')

Bad

func(1,2,axis='x',angle=90,size=450,name='foo bar')

5. Comments and Docstrings

5.1. Comments

Use inline comments sparingly and keep them concise.
Use docstrings to document modules, classes, methods, and functions.
Write clear, self-explanatory code and minimize the need for comments.

Don't Capture Change History/Log

Expand for more details

The change history SHOULD NOT be stored in the code itself.
Manually maintaining the change history in the code itself is unreliable, as it doesn't show the actual change that has occurred and isn't enforceable.
- The change history is captured by the version control system (e.g., Git) for each commit that's occurred. All changes that are pushed to development and production are captured, and diffs between those changes are available.
An example of the anti-pattern that SHOULD NOT be used:

# Created by: Joe Bloggs
# Date: 01-01-2021
# Purpose: Eg transformation script

# Change log
# Date: 02-01-2021
# Change Made By: Donald Duck
# Change Description: Small change to JSON file read in

5.2. Docstring

All functions should start with a Docstring, explaining the function, what it does, and why.

DocString Format: Use reST

Use the reST (restructuredText) Docstring format.

What is the reST (restructuredText) Docstring format?

Expand the menu below for more details.

Click to expand

One of the most prevalent Docstring formats today is reST (restructuredText). Shown below are the main components of the reST Docstring format:

"""[Summary]
:param [ParamName]: [ParamDescription], defaults to [DefaultParamVal]
:type [ParamName]: [ParamType](, optional)
...
:raises [ErrorType]: [ErrorDescription]
...
:return: [ReturnDescription]
:rtype: [ReturnType]
"""

For an overview of the reST Docstring format, see here.
The main components of the reST Docstring format are described below.

6. Error Handling

Prefer specific exceptions over catching all exceptions (except Exception:).
Use try-except-else blocks when possible.
Use finally blocks sparingly and only when necessary.

7. General Guidelines

Failing all of the above, follow the PEP 8 guidelines for conventions not covered here.
For additional inspiration, refer to the Google Python style guide for great examples of do's and don'ts on writing clean Python code.
Remember, consistency is key when it comes to style guides. Following the established style within a project or organization is essential to maintain code readability and maintainability. cd

paulf-999 / python_templates

Python Templates

Python Style Guide

1. Naming Conventions

1.1. Case Styles, Per-Python Object

Case Style Specification

1.2. Avoid Abbreviations

2. Code Layout

2.1. General

Indentation

Line length

Whitespace: imports, classes, and functions

Whitespace: methods and classes

Whitespace: operators and commas

Whitespace: code readability

2.2. Imports

3. String Formatting: Use `f-strings`

4. Function and Method Definitions

5. Comments and Docstrings

5.1. Comments

Don't Capture Change History/Log

5.2. Docstring

DocString Format: Use reST

6. Error Handling

7. General Guidelines

About

Languages

Python Templates

Python Style Guide

1. Naming Conventions

1.1. Case Styles, Per-Python Object

Case Style Specification

1.2. Avoid Abbreviations

2. Code Layout

2.1. General

Indentation

Line length

Whitespace: imports, classes, and functions

Whitespace: methods and classes

Whitespace: operators and commas

Whitespace: code readability

2.2. Imports

3. String Formatting: Use f-strings

4. Function and Method Definitions

5. Comments and Docstrings

5.1. Comments

Don't Capture Change History/Log

5.2. Docstring

DocString Format: Use reST

6. Error Handling

7. General Guidelines

About

Languages

3. String Formatting: Use `f-strings`