Giters

wahl-sec / doc-log

Python docstring parser with forgiving type checking and logging for multiple docstring dialects.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

pythonpython3documentationdocstringsparsing-librarytypechecker

doc-log

doc-log offers the ability to parse docstrings and ultimately make decisions based on parsed rules. Everything that doc-log can do is listed under the Features section. Follow the steps in Setup to install doc-log. Examples and use-cases are listed under the Examples section.

Features

Parsing

  • Parse docstrings based on any of the supported formats.
    • PEP257
    • Epytext
    • rEST
    • Google
    • numpydoc

Type Checking

  • Enforce runtime type checking based on the function/method docstring. Supports almost all of the types offered by the standard library typing.
    • PEP257
    • Epytext
    • rEST
    • Google
    • numpydoc

Logging

  • Log function/method calls based on docstring.

    • PEP257
    • Epytext
    • rEST
    • Google
    • numpydoc

  • Logging Information

    • Module that the function/method was called from.
    • Time of function/method call.
    • Passive type checking of passed parameters.
      • Warning level or below.
    • Active type checking of passed parameters and return types.
      • Error level or above.

Examples

Using doc-log

# add_two.py

from doc_log import doc_log


@doc_log(dialect="pep257", type_check=True, _active_type_check=True)
def add_two(i):
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Types:
    i -- int

    Returns:
    Integer `i` plus two (2)

    Return Type:
    int
    """
    return i + 2


print(f"{add_two(i=2)} == 4")
$ python3 add_two.py
WARNING :: 2021-08-27 19:33:45,415 :: (doc-log :: <module>:14:25) parameter: `i` had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
WARNING :: 2021-08-27 19:33:45,415 :: (doc-log :: <module>:20:25) return type had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
INFO :: 2021-08-27 19:33:45,415 :: (doc-log :: <module>:7:25) function: `add_two` called from `.../doc-log/add_two.py` at: `2021-08-27T19:33:45.415286`
4 == 4

Invalid Type Provided

# add_two.py

from doc_log import doc_log


@doc_log(dialect="pep257", type_check=True, _active_type_check=True)
def add_two(i):
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Types:
    i -- int

    Returns:
    Integer `i` plus two (2)

    Return Type:
    int
    """
    return i + 2


print(f"{add_two(i='2')} == 4")
$ python3 add_two.py
WARNING :: 2021-08-27 19:34:14,417 :: (doc-log :: <module>:14:25) parameter: `i` had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
WARNING :: 2021-08-27 19:34:14,417 :: (doc-log :: <module>:20:25) return type had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
Traceback (most recent call last):
  File ".../doc-log/add_two.py", line 25, in <module>
    print(f"{add_two(i='2')} == 4")
  File ".../doc-log/doc_log/__init__.py", line 48, in wrapper
    raise TypeError(
TypeError: (doc-log :: <module>:14:25) parameter: `i` was not of expected type: `int` was actually `str`

Invalid Type Defined

# add_two.py

from doc_log import doc_log


@doc_log(dialect="pep257", type_check=True, _active_type_check=True)
def add_two(i):
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Types:
    i -- str

    Returns:
    Integer `i` plus two (2)

    Return Type:
    int
    """
    return i + 2


print(f"{add_two(i=2)} == 4")
$ python3 add_two.py
WARNING :: 2021-08-27 19:34:39,279 :: (doc-log :: <module>:14:25) parameter: `i` had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `str`
WARNING :: 2021-08-27 19:34:39,279 :: (doc-log :: <module>:20:25) return type had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
Traceback (most recent call last):
  File ".../doc-log/add_two.py", line 25, in <module>
    print(f"{add_two(i=2)} == 4")
  File ".../doc-log/doc_log/__init__.py", line 48, in wrapper
    raise TypeError(
TypeError: (doc-log :: <module>:14:25) parameter: `i` was not of expected type: `str` was actually `int`

Logging With Type Check

# add_two.py

from logging import INFO, getLogger, basicConfig

basicConfig(format="%(levelname)s :: %(asctime)s :: %(message)s", level=INFO)
logger = getLogger(__name__)

from doc_log import doc_log


@doc_log(dialect="pep257", type_check=True)
def add_two(i, j=None):
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Types:
    i -- int

    Returns:
    Integer `i` plus two (2)

    Return Type:
    int
    """
    logger.info("parameter: `i` is {!s}".format(i))
    return i + 2


print(f"{add_two(i=2, j=3)} == 4")
$ python3 add_two.py
WARNING :: 2021-08-27 19:35:03,278 :: (doc-log :: <module>:19:31) parameter: `i` had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
WARNING :: 2021-08-27 19:35:03,279 :: (doc-log :: <module>:25:31) return type had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
WARNING :: 2021-08-27 19:35:03,292 :: (doc-log :: <module>:12:31) parameter: `j` was not of expected type: `_empty` was actually `int`
INFO :: 2021-08-27 19:35:03,293 :: (doc-log :: <module>:12:31) function: `add_two` called from `.../doc-log/add_two.py` at: `2021-08-27T19:35:03.292657`
INFO :: 2021-08-27 19:35:03,293 :: parameter: `i` is 2
4 == 4

Logging With Type Check (Error)

# add_two.py

from logging import ERROR, getLogger, basicConfig

basicConfig(format="%(levelname)s :: %(asctime)s :: %(message)s", level=ERROR)
logger = getLogger(__name__)

from doc_log import doc_log


@doc_log(dialect="pep257", type_check=True)
def add_two(i, j=None):
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Types:
    i -- int

    Returns:
    Integer `i` plus two (2)

    Return Type:
    int
    """
    logger.info("parameter: `i` is {!s}".format(i))
    return i + 2


print(f"{add_two(i=2, j=3)} == 4")
$ python3 add_two.py
Traceback (most recent call last):
  File ".../doc-log/add_two.py", line 31, in <module>
    print(f"{add_two(i=2, j=3)} == 4")
  File ".../doc-log/doc_log/__init__.py", line 48, in wrapper
    raise TypeError(
TypeError: (doc-log :: <module>:12:31) parameter: `j` was not of expected type: `_empty` was actually `int`

Logging With Type Check (Debug)

# add_two.py

from logging import DEBUG, getLogger, basicConfig

basicConfig(format="%(levelname)s :: %(asctime)s :: %(message)s", level=DEBUG)
logger = getLogger(__name__)

from doc_log import doc_log


@doc_log(dialect="pep257", type_check=True)
def add_two(i, j=None):
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Types:
    i -- int

    Returns:
    Integer `i` plus two (2)

    Return Type:
    int
    """
    logger.info("parameter: `i` is {!s}".format(i))
    return i + 2


print(f"{add_two(i=2, j=3)} == 4")
$ python3 add_two.py
DEBUG :: 2021-08-27 19:35:56,182 :: (doc-log :: <module>:12:31) parsing docstring with dialect: `pep257` from function: `add_two` in `.../doc-log/add_two.py`
DEBUG :: 2021-08-27 19:35:56,183 :: (doc-log :: <module>:12:31) parsed sections: `['arguments', 'types', 'returns', 'rtypes']` from function: `add_two` in `.../doc-log/add_two.py`
WARNING :: 2021-08-27 19:35:56,184 :: (doc-log :: <module>:19:31) parameter: `i` had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
WARNING :: 2021-08-27 19:35:56,184 :: (doc-log :: <module>:25:31) return type had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
DEBUG :: 2021-08-27 19:35:56,190 :: (doc-log :: <module>:19:31) item: `int` is not nested, type checking directly against value: `2`
DEBUG :: 2021-08-27 19:35:56,190 :: (doc-log :: <module>:12:31) item: `_empty` is not nested, type checking directly against value: `3`
DEBUG :: 2021-08-27 19:35:56,191 :: (doc-log :: <module>:18:31) type check arguments results was: `{'i': SectionItemTypeResult(item=SectionItem(value='int', _subitems=[], name='i', lineno=7), result=True, expected='int', actual='int', _subitems=[]), 'j': SectionItemTypeResult(item=SectionItem(value='_empty', _subitems=[], name='j', lineno=0), result=False, expected='_empty', actual='int', _subitems=[])}`
WARNING :: 2021-08-27 19:35:56,191 :: (doc-log :: <module>:12:31) parameter: `j` was not of expected type: `_empty` was actually `int`
INFO :: 2021-08-27 19:35:56,191 :: (doc-log :: <module>:12:31) function: `add_two` called from `.../doc-log/add_two.py` at: `2021-08-27T19:35:56.191936`
DEBUG :: 2021-08-27 19:35:56,191 :: (doc-log :: <module>:12:31) function: `add_two` was passed arguments: `()` and keyword arguments: `{'i': 2, 'j': 3}`
INFO :: 2021-08-27 19:35:56,191 :: parameter: `i` is 2
DEBUG :: 2021-08-27 19:35:56,196 :: (doc-log :: <module>:25:31) return type: `int` is not nested, type checking directly against value: `4`
DEBUG :: 2021-08-27 19:35:56,198 :: (doc-log :: <module>:24:31) type check return results was: `SectionItemTypeResult(item=SectionItem(value='int', _subitems=[], name=None, lineno=13), result=True, expected='int', actual='int', _subitems=[])`
4 == 4

Logging Without Type Check

# add_two.py

from logging import INFO, getLogger, basicConfig

basicConfig(format="%(levelname)s :: %(asctime)s :: %(message)s", level=INFO)
logger = getLogger(__name__)

from doc_log import doc_log


@doc_log(dialect="pep257", type_check=False)
def add_two(i, j=None):
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Types:
    i -- int

    Returns:
    Integer `i` plus two (2)

    Return Type:
    int
    """
    logger.info("parameter: `i` is {!s}".format(i))
    return i + 2


print(f"{add_two(i=2, j=3)} == 4")
$ python3 add_two.py
WARNING :: 2021-08-27 19:36:16,881 :: (doc-log :: <module>:19:31) parameter: `i` had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
WARNING :: 2021-08-27 19:36:16,881 :: (doc-log :: <module>:25:31) return type had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
INFO :: 2021-08-27 19:36:16,886 :: parameter: `i` is 2
4 == 4

Parsing PEP257 Docstring

# add_two.py

from doc_log.parser import parse_docstring

def add_two(i: int) -> int:
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Returns:
    Integer `i` plus two (2)
    """
    return i + 2

print(parse_docstring(add_two, dialect="pep257"))
$ python3 add_two.py
WARNING :: 2021-08-27 19:36:35,514 :: (doc-log :: <module>:7:18) parameter: `i` had different type hints in the docstring and in the signature, signature: `int` / docstring: `_empty`
WARNING :: 2021-08-27 19:36:35,514 :: (doc-log :: <module>:7:18) return type had different type hints in the docstring and in the signature, signature: `int` / docstring: `_empty`
{'arguments': Section(section='arguments', items=[SectionItem(value='the provided integer', _subitems=[], name='i', lineno=4)], _function=<code object add_two at 0x000002C883AD45B0, file ".../doc-log/add_two.py", line 6>, lineno=3), 'returns': Section(section='returns', items=[SectionItem(value='Integer `i` plus two (2)', _subitems=[], name=None, lineno=7)], _function=<code object add_two at 0x000002C883AD45B0, file ".../doc-log/add_two.py", line 6>, lineno=6), 'types': Section(section='types', items=[SectionItem(value='int', _subitems=[], name='i', lineno=0)], _function=<code object add_two at 0x000002C883AD45B0, file ".../doc-log/add_two.py", line 6>, lineno=0), 'rtypes': Section(section='rtypes', items=[SectionItem(value='int', _subitems=[], name=None, lineno=0)], _function=<code object add_two at 0x000002C883AD45B0, file ".../doc-log/add_two.py", line 6>, lineno=0)}

Parsing Type Hints in PEP257 Docstring

# add_two.py

from doc_log.parser import parse_docstring


def add_two(i):
    """Add two (2) to a provided integer and return.

    Arguments:
    i -- the provided integer

    Types:
    i -- int

    Returns:
    Integer `i` plus two (2)

    Return Type:
    int
    """
    return i + 2


print(parse_docstring(_function=add_two, dialect="pep257"))
$ python3 add_two.py
WARNING :: 2021-08-27 19:36:58,722 :: (doc-log :: <module>:14:24) parameter: `i` had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
WARNING :: 2021-08-27 19:36:58,722 :: (doc-log :: <module>:20:24) return type had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
{'arguments': Section(section='arguments', items=[SectionItem(value='the provided integer', _subitems=[], name='i', lineno=4)], _function=<code object add_two at 0x000001A0E25D45B0, file ".../doc-log/add_two.py", line 6>, lineno=3), 'types': Section(section='types', items=[SectionItem(value='int', _subitems=[], name='i', lineno=7)], _function=<code object add_two at 0x000001A0E25D45B0, file ".../doc-log/add_two.py", line 6>, lineno=6), 'returns': Section(section='returns', items=[SectionItem(value='Integer `i` plus two (2)', _subitems=[], name=None, lineno=10)], _function=<code object add_two at 0x000001A0E25D45B0, file ".../doc-log/add_two.py", line 6>, lineno=9), 'rtypes': Section(section='rtypes', items=[SectionItem(value='int', _subitems=[], name=None, lineno=13)], _function=<code object add_two at 0x000001A0E25D45B0, file ".../doc-log/add_two.py", line 6>, lineno=12)}

Parse Type Check Results in PEP257 Docstring

# add_two.py

from doc_log.parser import parse_docstring
from doc_log.types import type_check_arguments, type_check_rtypes


def add_two(i):
    """Add two (2) to a provided integer and return the original and the result.

    Arguments:
    i -- the provided integer

    Types:
    i -- int

    Returns:
    Integer `i` plus two (2)

    Return Type:
    Tuple[int, int]
    """
    return (i, i + 2)


parameters = {"i": 2}
result = add_two(**parameters)
docstring = parse_docstring(_function=add_two, dialect="pep257")

print("Arguments ==============")
print(type_check_arguments(docstring["types"], parameters=parameters))
print("Return ==============")
print(repr(type_check_rtypes(docstring["rtypes"], results=result)))
$ python3 add_two.py
WARNING :: 2021-08-27 19:37:24,673 :: (doc-log :: <module>:15:27) parameter: `i` had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `int`
WARNING :: 2021-08-27 19:37:24,673 :: (doc-log :: <module>:21:27) return type had different type hints in the docstring and in the signature, signature: `_empty` / docstring: `tuple[int, int]`
Arguments ==============
{'i': SectionItemTypeResult(item=SectionItem(value='int', _subitems=[], name='i', lineno=7), result=True, expected='int', actual='int', _subitems=[])}
Return ==============
SectionItemTypeResult(item=SectionItem(value='tuple', _subitems=[SectionItem(value='int', _subitems=[], name=None, lineno=13), SectionItem(value='int', _subitems=[], name=None, lineno=13)], name=None, lineno=13), result=True, expected='tuple', actual='tuple', _subitems=[SectionItemTypeResult(item=SectionItem(value='int', _subitems=[], name=None, lineno=13), result=True, expected='int', actual='int', _subitems=[]), SectionItemTypeResult(item=SectionItem(value='int', _subitems=[], name=None, lineno=13), result=True, expected='int', actual='int', _subitems=[])])

Setup

doc-log does not use any external dependency (except for pytest and black when developing)

git clone https://github.com/wahl-sec/doc-log.git && cd doc-log && pip install .

About

Python docstring parser with forgiving type checking and logging for multiple docstring dialects.

pythonpython3documentationdocstringsparsing-librarytypechecker

License:GNU General Public License v3.0

Languages

Language:Python 100.0%