Victoriapm / whale

🐳 The stupidly simple data discovery tool.

Home Page:https://docs.whale.cx

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

The simplest way to keep track of your warehouse tables

whale is a lightweight data discovery and documentation engine.

  • Automatically index all of the tables in your warehouse as plain markdown files -- so they're easily versionable, searchable, and editable either locally or through a remote git server.
  • Search for tables and documentation through the CLI or through a git remote server like Github.
  • Define and schedule basic metric calculations.

😁 Join the discussion on slack.


codecov slack

For a live demo, check out dataframehq/whale-bigquery-public-data.

πŸ“” Documentation

Read the docs for a full overview of whale's capabilities.

Installation

Mac OS

brew install dataframehq/tap/whale

All others

Clone this directory and run the following in the base directory of the repo:

make && make install

If you are running this multiple times, make sure ~/.whale/libexec does not exist, or your virtual environment may not rebuild. We don't explicitly add an alias for the whale binary, so you'll want to add the following alias to your .bash_profile or .zshrc file.

alias wh=~/.whale/bin/whale

Getting started

Setup

For individual use, run the following command to go through the onboarding process. It will (a) set up all necessary files in ~/.whale, (b) walk you through cron job scheduling to periodically scrape metadata, and (c) set up a warehouse:

wh init

The cron job will run as you schedule it (by default, every 6 hours). If you're feeling impatient, you can also manually run wh etl to pull down the latest data from your warehouse.

For team use, see the docs for instructions on how to set up and point your whale installation at a remote git server.

Seeding some sample data

If you just want to get a feel for how whale works, remove the ~/.whale directory and follow the instructions at dataframehq/whale-bigquery-public-data.

Go go go!

Run:

wh

to search over all metadata. Hitting enter will open the editable part of the docs in your default text editor, defined by the environmental variable $EDITOR (if no value is specified, whale will use the command open).

ezoic increase your site revenue

About

🐳 The stupidly simple data discovery tool.

https://docs.whale.cx

License:GNU General Public License v3.0


Languages

Language:Python 75.0%Language:Rust 24.6%Language:Makefile 0.4%