JakeTRogers / goDiffIt

Compare files using boolean set operations

goDiffIt

Description

goDiffIt is a command-line utility written in Go that uses boolean operations(difference, intersection, and union) to compare 2 files. It can be used to compare text files, CSV files, or any other file that can be read as a series of lines. It can also read from file descriptors, so you can use it to compare the output of other commands.

It is different from the diff command in that it does not compare the contents of the files line-by-line. Instead, it compares the files as sets of lines. This means that it can be used to compare files that have the same lines in different orders, or files that have the same lines repeated multiple times.

A common use case for this tool is comparing lists of servers from different sources to see which servers are missing from one or the other. Alternatively, after looping over a few thousand systems to return configuration data in CSV format, this tool can identify any systems that didn't return results.

Installation

1. Download the binary for your preferred platform from the releases page
2. Extract the archive. It contains this readme, a copy of the Apache 2.0 license, and the goDiffIt binary.
3. Copy the binary to a directory in your $PATH. i.e. /usr/local/bin

Usage

To use goDiffIt, you need to pass two arguments: the paths to the two files you want to compare:

./godiffit fileA.txt fileB.txt

This will print lines that appear in fileA.txt but not in fileB.txt, and lines that appear in fileB.txt but not in fileA.txt.

If you're comparing CSV files, you can specify the delimiter with the --delimiter flag:

./godiffit --delimiter=";" fileA.csv fileB.csv

The tool can also read from file descriptors:

./godiffit <(cut -d, -f2,3 fileA) <(grep -v '^#' fileB)

Examples

If fileA.txt contains:

apple
banana
cherry
orange

and fileB.txt contains:

BANANA.example.com,abc,xyz
Cherry,stuff, and, things
date

goDiffIt fileA.txt fileB.txt --ignore-fqdn will output:

Difference of fileA.txt - fileB.txt:
apple
orange

Difference of fileB.txt - fileA.txt:
date

goDiffIt fileA.txt fileB.txt --intersection --ignore-fqdn will output:

Intersection of fileA.txt and fileB.txt:
banana
cherry

goDiffIt fileA.txt fileB.txt --union --ignore-fqdn will output:

Union of fileA.txt and fileB.txt:
apple
banana
cherry
date
orange