hueristiq / xstream

A versatile command-line interface (CLI) utility for handling data streams in Unix-like environments. It serves as a crucial link between standard input and dual outputs, taking a single stream of input and splitting it into two outputs, one to the standard output and the other to a file simultaneously.

xstream

xstream is a versatile command-line interface (CLI) utility for handling data streams in Unix-like environments. It serves as a crucial link between standard input and dual outputs, taking a single stream of input and splitting it into two outputs, one to the standard output and the other to a file simultaneously.

Resources

• Features
• Installation
  • Install release binaries (Without Go Installed)
  • Install source (With Go Installed)
    • go install ...
    • go build ... the development Version
• Usage
  • Examples
    • Appending Unique Lines to File
    • Deduplicating Files
• Contributing
• Licensing
• Credits
  • Contributors
  • Similar Projects

Features

• Writes incoming stdin to both stdout and file.
• Supports reading and writing to the same file.
• Supports appending and overwriting destination.
• Supports deduplication.
• Cross-Platform (Windows, Linux & macOS).

Installation

Install release binaries (Without Go Installed)

Visit the releases page and find the appropriate archive for your operating system and architecture. Download the archive from your browser or copy its URL and retrieve it with wget or curl:

• ...with wget:

   wget https://github.com/hueristiq/xstream/releases/download/v<version>/xstream-<version>-linux-amd64.tar.gz

• ...or, with curl:

   curl -OL https://github.com/hueristiq/xstream/releases/download/v<version>/xstream-<version>-linux-amd64.tar.gz

...then, extract the binary:

tar xf xstream-<version>-linux-amd64.tar.gz

Tip

The above steps, download and extract, can be combined into a single step with this onliner

curl -sL https://github.com/hueristiq/xstream/releases/download/v<version>/xstream-<version>-linux-amd64.tar.gz | tar -xzv

Note

On Windows systems, you should be able to double-click the zip archive to extract the xstream executable.

...move the xstream binary to somewhere in your PATH. For example, on GNU/Linux and OS X systems:

sudo mv xstream /usr/local/bin/

Note

Windows users can follow How to: Add Tool Locations to the PATH Environment Variable in order to add xstream to their PATH.

Install source (With Go Installed)

Before you install from source, you need to make sure that Go is installed on your system. You can install Go by following the official instructions for your operating system. For this, we will assume that Go is already installed.

go install ...

go install -v github.com/hueristiq/xstream/cmd/xstream@latest

go build ... the development Version

• Clone the repository

   git clone https://github.com/hueristiq/xstream.git 

• Build the utility

   cd xstream/cmd/xstream && \
   go build .

• Move the xstream binary to somewhere in your PATH. For example, on GNU/Linux and OS X systems:

   sudo mv xstream /usr/local/bin/

  Windows users can follow How to: Add Tool Locations to the PATH Environment Variable in order to add xstream to their PATH.

Caution

While the development version is a good way to take a peek at xstream's latest features before they get released, be aware that it may have bugs. Officially released versions will generally be more stable.

Usage

To display help message for xstream use the -h flag:

xstream -h

help message:

          _
__  _____| |_ _ __ ___  __ _ _ __ ___
\ \/ / __| __| '__/ _ \/ _` | '_ ` _ \
 >  <\__ \ |_| | |  __/ (_| | | | | | |
/_/\_\___/\__|_|  \___|\__,_|_| |_| |_|
                                 v0.3.0

       with <3 by Hueristiq Open Source

USAGE:
 xstream [OPTIONS]

INPUT:
     --soak bool        soak up all input before writing to file

MANIPULATION:
     --trim bool        enable leading and trailing whitespace trimming

OUTPUT:
 -u, --unique bool      output unique lines
 -a, --append bool      append lines to output
 -q, --quiet bool       suppress output to stdout
 -p, --preview bool     preview new lines, without writing to file

pflag: help requested

Examples

Appending Unique Lines to File

➜  cat stuff.txt
one
two
three

➜  cat new-stuff.txt
zero
one
two
three
four
five

➜  cat new-stuff.txt | xstream stuff.txt --append --unique
zero
four
five

➜  cat stuff.txt
one
two
three
zero
four
five

Note that the new lines added to stuff.txt are also sent to stdout, this allows for them to be redirected to another file:

➜  cat new-stuff.txt | xstream stuff.txt --append --unique > added-lines.txt
➜  cat added-lines.txt
zero
four
five

Deduplicating Files

➜  cat stuff.txt
zero
one
two
three
zero
four
five
five

➜  cat stuff.txt | xstream stuff.txt --soak --unique
zero
one
two
three
four
five

➜  cat stuff.txt
zero
one
two
three
four
five

Note the use of --soak, it makes the utility soak up all its input before writing to a file. This is useful for reading from and writing to the same file in a single pipeline.

Contributing

Issues and Pull Requests are welcome! Check out the contribution guidelines.

Licensing

This tool is distributed under the MIT license.

Credits

Contributors

Thanks to the amazing contributors for keeping this project alive.

Similar Projects

Thanks to similar open source projects - check them out, may fit in your workflow.

anew ◇ tee (coreutils) ◇ sponge (moreutils)