NAME

App::Netsync - network/database synchronization utility/library

DESCRIPTION

netsync is a network synchronization tool that:

- maps network interfaces to their respective (potentially stacked) devices
- gathers interface-specific information from an asset management database
- sends the information it gathers to each device
- provides ways of producing useful information about the network

Note: All communication with network nodes is done using SNMP.

Overview

In a typical run, execution begins by parsing a configuration file (-c). Next, Netsync searches for active network devices at each node. Nodes may be specified by filename, DNS (-D), or the keyboard (STDIN). Furthermore, nodes may be required to match a hostname pattern (-m). Gathered information may be used to identify each device in a database. A database may be specified using Database (-d) or DB in the configuration file. Identified devices may then be updated (-u) with interface-specific information.

See doc/algorithm.svg for corresponding visual guidance (/usr/share/netsync/algorithm.svg after installation).

0 Installation

$ cpan App::Netsync

$ perl Makefile.PL
$ make
$ make test
$ make install

Modules

App::Netsync::Configurator: configuration file support
App::Netsync::Network: methods for manipulating network structures
App::Netsync::Scribe: I/O framework
App::Netsync::SNMP: SNMP framework

1 Invocation

After installation, netsync may be invoked by name:

$ netsync

If installation is skipped, netsync may be manually invoked as follows:

$ perl -I lib script/netsync

Options

-h --help

Help. Print usage and options.

Note: Help and Version print information and exit. Normal execution is skipped when either of these options are used.

-V

Version. Print build information.

Note: Help and Version print information and exit. Normal execution is skipped when either of these options are used.

-v

Verbose. Print everything.

Note: Quiet mode overrides Verbose mode.

-q

Quiet. Print nothing.

-c .ini

Config. Specify a configuration file.

default: /etc/netsync/netsync.ini

-p #

Probe. There are 2 Probe levels.

If the Probe option is used, resources will be created to aid in future runs.

Note: Each level corresponds to a stage of execution and is accumulative.

Probe Level 1: Probe the network for active nodes. See the Probe1Cache setting.
Probe Level 2: Probe the database for those nodes. See the Probe2Cache setting.

-D

DNS. Use your network's domain name system to retrieve a list of nodes.

Note: If neither DNS (-D) nor Nodes are specified, input will be read from standard input (STDIN). If both are present, Nodes is used.

-m pattern

Match. Only discover nodes with hostnames matching the given pattern.

-d .csv

Database. Specify an RFC4180-compliant database file.

-a

Automatch. Enable interface auto-matching.

Note: Automatch will be helpful if the database manages interfaces numerically. It aligns numerical ports (e.g. 23) in the database with ifNames (e.g. ethernet23 or Gi1/0/23) on the device.

-u

Update. Send interface-specific information to network nodes.

Parameters

nodes

Nodes. Specify an RFC1035-compliant list of network nodes.

Note: If neither DNS (-D) nor Nodes are specified, input will be read from standard input (STDIN). If both are present, Nodes is used.

2 Configuration

A configuration file may be specified using the Config (-c) option. Otherwise, a generic configuration file (/etc/netsync/netsync.ini) is used, but it will most likely require additional information before running properly. Namely, the following settings must be provided for a typical installation:

DNS

Required

Note: These settings are unnecessary if only the Nodes parameter is used.

domain: network domain name (e.g. example.com)

Optional

See Net::DNS for more options.

SNMP

Required

Note: SNMP settings are unnecessary if a default SNMP environment is used; however, this is HIGHLY insecure and discouraged.

Version

SNMPv3 (recommended)

SecLevel

the level of security to deploy

Security Levels

noAuthNoPriv (default): This causes SNMPv3 to mimic v2!
authNoPriv
authPriv

SecName

username

default: initial

AuthPass

the access (authentication) key

PrivPass

the privacy (encryption) key

SNMPv2

Community

the SNMP community to address

default: public

Optional

See SNMP::Session for more options.

DB

Optional

See DBI for more options.

Netsync

Synopsis

Table
 ---------------------------------------------------------
 |  DeviceField  |  InterfaceField  |  InfoFields...     |
 ---------------------------------------------------------         =============
 |   (serial)    |     (ifName)     |(interface-specific)|  --->   || SyncOID ||
 |                          ...                          |         =============
 ---------------------------------------------------------              (device)

Note: If the Update (-u) option is used, netsync will overwrite the device with the corresponding InfoFields.

Required

DBMS: the type of database (i.e. a Perl DBD, e.g. Pg, mysql, Oracle, etc.).
Server: the database network location
Port: the database transport location
Database: the name of the database to use
Username: the name of a user that has access to the database
Password: the authentication key of the user

Note: Settings above are unnecessary if only the Database (-d) option is used.

MIBdir

the location of required MIBs

default: /usr/share/netsync/mib/

Required MIBs

standard

IF-MIB
ENTITY-MIB

Cisco

CISCO-STACK-MIB

Brocade

FOUNDRY-SN-AGENT-MIB
FOUNDRY-SN-SWITCH-GROUP-MIB

HP

SEMI-MIB

Table: the name of the table in the database that contains the following fields
DeviceField: the field that provides a unique ID for each device (typically a serial number)
InterfaceField: the field that holds interface names retrieved from the IF-MIB (ifName) via SNMP
InfoFields: a comma-separated list of fields containing interface-specific information

Optional

SyncOID: the MIB location to store interface-specific information via SNMP

See netsync.pl and Netsync.pm for more information.

3 Structures

Netsync builds an internal view of the network hardware whenever it is used. Each node is associated with its IP address and device(s). Each device is associated with its serial number and interface(s). Each interface is associated with its ifName and interface-specific information.

The resulting data structure could be described as a list of trees.

|-> node (IP)
|-> node (IP)
|-> node (IP)
|                              -interface (ifName)
|                             /
|             -device (serial)--interface (ifName)
|            /                \
|-V node (IP)                  -interface (ifName)
|            \
|             -device (serial)--interface (ifName)
|                             \
|                              -interface (ifName)
|-> node (IP)
|-> node (IP)
|                              -interface (ifName)
|                             /
|-V node (IP)--device (serial)--interface (ifName)
|                             \
|                              -interface (ifName)
|-> node (IP)
|-> node (IP)
|-> node (IP)
|-> node (IP)
...

States

inactive: unreachable or unresponsive on the network
active: reachable and responsive on the network
unrecognized: active but not processable
recognized: active and processable
unidentified: recognized but not in the database
identified: recognized and in the database

4 Discovery

The first task netsync has is to find all relevant nodes on the network. Relevant nodes are specified one of three ways:

DNS (-D) option

A zone transfer is filtered by Netsync, and a list of A or AAAA records with valid IPv4 or IPv6 addresses is produced.

$ netsync -D

Nodes parameter

Nodes is a path to a file containing an RFC1035-compliant list of nodes.

$ netsync -Dqp1
$ netsync /var/cache/netsync/active.txt

STDIN (e.g. pipe or keyboard)

If neither -D nor Nodes are present, a node list is pulled from standard input. This allows pipelining with other utilities for extended functionality.

$ dig axfr example.com | grep hub123 | netsync

$ cat superset.txt | grep hub[0-9]+ | netsync

Note: Match (-m) may be used with any of the methods above.

$ netsync -Dm "(sw|hub)[0-9]+"

$ netsync -Dqp1
$ netsync -m "(sw|hub)[0-9]+" /var/cache/netsync/active.txt

$ dig axfr example.com | netsync -m "(sw|hub)[0-9]+"

www.example.com            <-- no match (www)
hub123.example.com         <-- match (hub123)
sw1234.example.com         <-- match (sw1234)
...

About RFC1035

RFC1035 (3.2.1) specifies a format for resource records served by name servers. Thus, for compatibility reasons, this application is RFC1035-compliant.

name             ttl class type address
www.example.com. 300 IN    A    93.184.216.119
www.example.com. 300 IN    AAAA 2606:2800:220:6d:26bf:1447:1097:aa7
...

5 Recognition

netsync must attempt to contact each specified node to see if it is active. Any node that netsync attempts to contact is logged in NodeLog. If a node is active, netsync will try to retrieve its device serial number(s). If multiple devices are discovered, netsync will try to map each interface to its device (serial). If no devices are discovered (or mapping is impossible), The node is considered unrecognized and skipped.

Note: Only ASCII serials are supported.

Supported Vendors

Brocade (stacks)
Cisco (stacks)
HP

Interface Mapping

1 Extract interfaces.

Standard

1 ifType (1.3.6.1.2.1.2.2.1.3)

Excluded ifTypes

1 other
24 softwareLoopback
53 propVirtual

2 ifDescr (1.3.6.1.2.1.2.2.1.2)

Note: ifName (1.3.6.1.2.1.31.1.1.1) may be an alternate.

2 Extract serials.

Standard

1 entPhysicalClass (1.3.6.1.2.1.47.1.1.1.1.5)

Included entPhysicalClasses

3 chassis

2 entPhysicalSerialNum (1.3.6.1.2.1.47.1.1.1.1.11)

Proprietary

Brocade

snChasUnitSerNum : 1.3.6.1.4.1.1991.1.1.1.4.1.1.2

Note: snChasSerNum (1.3.6.1.4.1.1991.1.1.1.1.2) may be an alternate; however, it does NOT support stacks.

Cisco

moduleSerialNumber : 1.3.6.1.4.1.9.5.1.3.1.1.3

Note: moduleSerialNumberString (1.3.6.1.4.1.9.5.1.3.1.1.26) may be an alternate.

HP

hpHttpMgSerialNumber : 1.3.6.1.4.1.11.2.36.1.1.2.9

3 Map interfaces to serials.

Standard

Note: There is no known standard for achieving this.

Proprietary

Cisco

1 portIfIndex

1.3.6.1.4.1.9.5.1.4.1.1.11

2 portModuleIndex

1.3.6.1.4.1.9.5.1.4.1.1.1

3 moduleSerialNumber

1.3.6.1.4.1.9.5.1.3.1.1.3

Note: moduleSerialNumberString (1.3.6.1.4.1.9.5.1.3.1.1.26) may be an alternate.

Brocade

1 snSwPortIfIndex

1.3.6.1.4.1.1991.1.1.3.3.1.1.38

2 snSwPortDescr

1.3.6.1.4.1.1991.1.1.3.3.1.1.39

Note: Unit/Module/Interface (U/M/I) is assumed to map Unit to module IID.

3 snChasUnitSerNum

1.3.6.1.4.1.1991.1.1.1.4.1.1.2

6 Probe Level 1

Probe Level 1 is specified using -p1 and updates Probe1Cache.

During Probe Level 1, only the discovery stage is executed. It caches discovered nodes in Probe1Cache. This cache is created in RFC1035-complaint format. It may subsequently be passed as the Nodes parameter to skip inactive nodes.

$ netsync -Dm "sw[^.]+|hub[0-9]+" -p1
> configuring (using /etc/netsync/netsync.ini)...
> discovering (using DBMS)...  480 nodes (10 skipped), 500 devices
$ netsync /var/cache/netsync/active.txt
> configuring (using /etc/netsync/netsync.ini)...
> discovering (using /var/cache/netsync/active.txt)...  480 nodes, 500 devices
> identifying (using DBMS)...  498 synchronized (4 conflicts)

Note: Since the goal of Probe Level 1 is to save time later, unrecognized nodes will not be included in Probe1Cache.

7 Identification

Once netsync has a view of the network's hardware, it requires a database to find information specific to each interface.

Note: Since netsync treats the database as read-only, the table and fields must already be present in the database.

The database may be provided one of two ways:

DBMS: This must be preconfigured in the configuration file and on the DBMS.
Database (-d): An RFC4180-compliant database file may be specified.

netsync identifies the network interface referenced in each database entry. A valid entry must not have missing DeviceField or InterfaceField information. Valid rows are synchronized with the network while invalid rows are skipped. Entries with unknown (not found) devices are skipped. Entries are then checked for conflicts.

About RFC4180

RFC4180 specifies a simple format (CSV) for database files. This format is widely supported making it useful for importing and exporting. Thus, for compatibility, this application is RFC4180-compliant.

DeviceField,InterfaceField,InfoField
A1B2C3D4E5F6,ethernet1/1/1,PoE Enabled
A1B2C3D4E5F6,ethernet1/2/1,Uplink
...

8 Synchronization

All conflicts are logged in ConflictLog and skipped.

Conflicts

Mismatch: This occurs when an InterfaceField value can't be found on an identified device.
Duplicate: This occurs when more than 1 entry for an interface exists in the database.
Unidentified: This occurs when hardware is found on the network but not in the database.

9 Probe Level 2

Probe level 2 is specified using -p2 and updates Probe1Cache and Probe2Cache.

During Probe Level 2, only the discovery and identification stages are executed. It caches identified nodes in Probe2Cache. This cache is created in RFC4180-complaint format. It may be passed as Nodes in subsequent runs to skip synchronization.

$ netsync -Dm "host[0-9]+" -a -p2
> configuring (using /etc/netsync/netsync.ini)... done
> discovering (using DNS)...  480 nodes (10 skipped), 500 devices
> identifying (using DBMS)...  498 synchronized (4 conflicts)
> Do you want to resolve conflicts now? [y/n] n
$ netsync -d /var/cache/netsync/synced.csv /var/cache/netsync/active.txt
> configuring (using /etc/netsync/netsync.ini)... done
> discovering (using /var/cache/netsync/active.txt)...  480 nodes, 500 devices
> identifying (using /var/cache/netsync/synced.csv)...  500 synchronized

Note: All unidentified hardware will be present in Probe2Cache. This is so the output of Probe Level 2 can serve as a network snapshot.

10 Updating

All modifications made to any device are logged in UpdateLog.

If probing is not used and the Update (-u) option is specified, netsync attempts to actualize its internally synchronized network using SNMP. It pushes gathered interface-specific information to the devices on the network. This information is stored in the device at the specified SyncOID, and is overwritten anytime netsync updates it.

$ netsync -Dau

EXAMPLES

$ netsync -Dm "sw[^.]+|hub[0-9]+" -au
> configuring (using /etc/netsync/netsync.ini)...
> discovering (using DNS)...  780 nodes (50 skipped), 800 devices (10 stacks)
> identifying (using DBMS)...  670 synchronized (4 conflicts)
> updating... 670 successful

$ dig axfr domain.tld | egrep ^(sw[^.]+|hub[0-9]+) | netsync -a
> configuring (using /etc/netsync/netsync.ini)...
> discovering (using STDIN)...  780 nodes (50 skipped), 800 devices
> identifying (using DBMS)...  670 synchronized (4 conflicts)

$ netsync -Dm "sw[^.]+|hub[0-9]+" -p1
> configuring (using /etc/netsync/netsync.ini)...
> discovering (using DBMS)...  780 nodes (50 inactive), 800 devices
$ netsync /var/cache/netsync/active.txt
> configuring (using /etc/netsync/netsync.ini)...
> discovering (using /var/cache/netsync/active.txt)...  780 nodes, 800 devices
> identifying (using DBMS)...  670 synchronized (4 conflicts)

$ netsync -Dm "sw[^.]+|hub[0-9]+" -a -p2
> configuring (using /etc/netsync/netsync.ini)...
> discovering (using DNS)...  780 nodes (50 skipped), 800 devices
> identifying (using DBMS)...  670 synchronized (4 conflicts)
$ netsync -d /var/cache/netsync/synced.csv /var/cache/netsync/active.txt
> configuring (using /etc/netsync/netsync.ini)...
> discovering (using /var/cache/netsync/active.txt)...  780 nodes, 800 devices
> identifying (using /var/cache/netsync/synced.csv)...  800 synchronized

AUTHOR

David Tucker, <dmtucker at ucsc.edu>

BUGS

Please report any bugs or feature requests to bug-netsync at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=App-Netsync. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

perldoc App::Netsync

You can also look for information at:

RT: CPAN's request tracker (report bugs here)

http://rt.cpan.org/NoAuth/Bugs.html?Dist=App-Netsync
AnnoCPAN: Annotated CPAN documentation

http://annocpan.org/dist/App-Netsync
CPAN Ratings

http://cpanratings.perl.org/d/App-Netsync
Search CPAN

http://search.cpan.org/dist/App-Netsync/

LICENSE

This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.

See http://dev.perl.org/licenses/ for more information.

dmtucker / netsync

NAME

DESCRIPTION

0 Installation

1 Invocation

2 Configuration

3 Structures

4 Discovery

5 Recognition

6 Probe Level 1

7 Identification

8 Synchronization

9 Probe Level 2

10 Updating

EXAMPLES

AUTHOR

BUGS

SUPPORT

LICENSE

About

Languages