netutils

A Python library that is a collection of objects for common network automation tasks.

This library intends to keep the following tenets:

Must not be any dependencies required to run the library.
- May be some optional dependencies, to be managed by the user in opt in fashion.
Shall prefer functions over classes.
Shall prefer a folder and file structure that is flat.
Shall leverage docstrings as the primary documentation mechanism.
- Must provide examples in every public function.
Shall retain a high test coverage.

Function Groupings

Functions are grouped with like functions, such as IP or MAC address based functions. Included to date are groupings of:

BGP ASN - Provides the ability to convert BGP ASN from integer to dot notation.
Configuration
- Cleaning - Provides the ability to remove or replace lines based on regex matches.
- Compliance - Provides the ability to compare two configurations to sanely understand the differences.
- Parsing - Provides the ability to parse configuration for the minor differences that are there.
DNS - Provides the ability to work with DNS, such as validating that a FQDN is resolvable.
Interface - Provides the ability to work with interface names, expanding, abbreviating, and spliting the names.
IP Address - Provides the ability to work with IP addresses, primarily exposing Python ipaddress functionality.
Library Mapper - Provides mappings in expected vendor names between Netmiko, NAPALM, pyntc, ntc-templates, pyats, and scrapli.
MAC Address - Provides the ability to work with MAC addresses such as validating or converting to integer.
Password - Provides the ability to compare and encrypt common password schemas such as type5 and type7 Cisco passwords.
Ping - Provides the ability to ping, currently only tcp ping.
Protocol Mapper - Provides a mapping for protocol names to numbers and vice versa.
Route - Provides the ability to provide a list of routes and an IP Address and return the longest prefix matched route.
VLANs - Provide the ability to convert configuration into lists or lists into configuration.

Examples

While all functions come with examples in the docstrings, for quick reference of the types of problems this library intends to solve the following examples are provided.

The following function will help in deploying list of VLANs and match the configuration style in a standard IOS-like configurations.

>>> from netutils.vlan import vlanlist_to_config
>>>
>>> vlan_cfg = vlanlist_to_config([1, 2, 3, 5, 6, 1000, 1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016, 1018])
>>>
>>> vlan_cfg
['1-3,5,6,1000,1002,1004,1006,1008,1010,1012,1014', '1016,1018']
>>>
>>> for index, line in enumerate(vlan_cfg):
...     if index == 0:
...         print(f"  switchport trunk allowed vlan {line}")
...     else:
...         print(f"  switchport trunk allowed vlan add {line}")
... 
  switchport trunk allowed vlan 1-3,5,6,1000,1002,1004,1006,1008,1010,1012,1014
  switchport trunk allowed vlan add 1016,1018
>>>

You may want to compare a known password with a given encrypted password. This can help in verifying if the passwords are as expected for compliance reasons.

>>> from netutils.password import compare_type5
>>>
>>> compare_type5("cisco","$1$nTc1$Z28sUTcWfXlvVe2x.3XAa.")
True
>>>
>>> compare_type5("not_cisco","$1$nTc1$Z28sUTcWfXlvVe2x.3XAa.")
False
>>>

Often times interfaces will come in various different shortened names, and it is helpful to normalize them.

>>> from netutils.interface import canonical_interface_name
>>>
>>> canonical_interface_name("Gi1/0/1")
'GigabitEthernet1/0/1'
>>>
>>> canonical_interface_name("Eth1")
'Ethernet1'
>>>

These are just some examples of the many functions provided by this library.

Attribution

The library was built to be a centralized place for common network automation code to be accessed. While in most cases it is difficult, if not impossible to understand the origin of code, the following intends to describe the known motivation for where code was derived from and in the few cases where actual code was directly taken from. Except where noted, all code is believed to be unattributable, from @itdependsnetworks, or from another Network to Code employee. If this is in fact an error, please open an issue, and the proper attribution will be provided. Any errors were not done out of malice, but rather the natural developer workflow of pulling snippets of code from existing locations such as StackOverflow and Github over months and years of development. As an example it is nearly impossible to understand the original author of Cisco type7 encryption/decryption in the sea of available code, and remains unattributable though clearly originally developed prior to this library being created.

Influencers

In many instances variables and function names were reused, but the code was built from scratch to avoid any potential licensing issues. Functions that were known to be rewritten and their known origin.

Function	Origin
asn_to_int	NAPALM
is_ip	IPCal
ip_to_bin	IPCal
get_usable_range	IPCal
encrypt_type7	unknown
decrypt_type7	unknown
vlan_to_list	Ansible
sanitize_config	NAPALM

Relevant PR's

Contributing

Pull requests are welcomed and automatically built and tested against multiple versions of Python through TravisCI.

The project is packaged with a light development environment based on docker-compose to help with the local development of the project and to run tests within TravisCI.

The project is following Network to Code software development guidelines and are leveraging the following:

Black, Pylint, Bandit, flake8, and pydocstyle for Python linting and formatting.
pytest, coverage, and unittest for unit tests.

CLI Helper Commands

The project features a CLI helper based on invoke to help setup the development environment. The commands are listed below in 3 categories:

dev environment
utility
testing.

Each command can be executed with invoke <command>. Each command also has its own help invoke <command> --help

Local dev environment

  build            Build all docker images.
  clean            Remove the project specific image.
  rebuild          Clean the Docker image and then rebuild without using cache.

Utility

  cli              Enter the image to perform troubleshooting or dev work.

Testing

  bandit           Run bandit to validate basic static code security analysis.
  black            Run black to check that Python files adhere to its style standards.
  coverage         Run the coverage report against pytest.
  flake8           Run flake8 to check that Python files adhere to its style standards.
  pylint           Run pylint code analysis.
  pydocstyle       Run pydocstyle to validate docstring formatting adheres to NTC defined standards.
  tests            Run all tests for the specified name and Python version.
  yamllint         Run yamllint to validate formatting adheres to NTC defined YAML standards.

Questions

For any questions or comments, feel free to swing by the Network to Code slack channel (channel #networktocode). Sign up here

josh-silvas / netutils