davidnewhall / indigraphs

Graphs for Indigo Data from SQL Logger Plugin

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Deprecated

Indigraphs

Graphs for Indigo Data.

Here you will find detailed instructions for how to graph every piece of data Indigo creates using free software: PostgreSQL, Graphite and Grafana. Also provided is a custom script you can schedule in Indigo. It copies your SQL data into graphite.

Step 1 - Get your data into SQL.

Download PostgresAPP from http://postgresapp.com

  • It doesn't require much configuration.
  • Set it to turn on at login so it's always available.

Setup the Indigo SQL Logger Plugin.

  • There are directions on Indigo's Wiki
  • Leave the database name as indigo_history to make things easy.
  • Event log entries are ignored, so turn them on if you need them.

Step 2 - Get Graphite running

Getting graphite running on your Mac is easy, but it does have a number of dependencies. Since you're running Indigo, and you are reading a github entry, I'm going to assume you either know how to get graphite running (go google) or you are ready to throw down and get these dependencies in order.

  • Install Homebrew:
  • /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • I'm not going to repeat the directions I followed. I'll just link to them and explain what I may have done differently.
  • I followed this pretty much exactly. Follow these steps to install graphite on OS X Mavericks.
  • If you don't have pip: sudo easy_install pip

You probably want carbon and graphite to auto-run on startup, right?

  • For GrahiteWeb (Django instance), I created a LaunchAgent plist to make OS X start it up. Copy these contents to /Library/LaunchAgents/pro.sleepers.graphite_launcher.plist
  • You also need to do mkdir /opt/graphite/log
  • Be sure to correct the UserName key! I recommend the same username as where you run Indigo, keep it simple.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>pro.sleepers.graphite_launcher</string>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StartInterval</key>
    <integer>600</integer>
    <key>StandardOutPath</key>
    <string>/opt/graphite/log/launchctl-carbon.stdout</string>
    <key>StandardErrorPath</key>
    <string>/opt/graphite/log/launchctl-carbon.stderr</string>
    <key>UserName</key>
    <string>administrator</string>
    <key>GroupName</key>
    <string>staff</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/django-admin.py</string>
        <string>runserver</string>
        <string>--pythonpath</string>
        <string>/opt/graphite/webapp</string>
        <string>--settings</string>
        <string>graphite.settings</string>
        <string>0.0.0.0:8088</string>
    </array>
    <key>WorkingDirectory</key>
    <string>/opt/graphite</string>
  </dict>
</plist>
  • Run this command to start it up:
sudo launchctl load -w /Library/LaunchAgents/pro.sleepers.graphite_launcher.plist
  • For Carbon, it has a start/stop/status script, so I just configure a schedule in Indigo to check it every 5 minutes. It is started if it happens to not be running. You could do considerably fancier things, like turn it into a virtual device you can turn on and off and graph and all that jazz, but meh. Just add the schedule like you see in these screen shots. Schedule for every 5 minutes.

Add a conditional script.

  • Here is the snippet of AppleScript you should copy into the Condition tab:
set ReturnCode to do shell script "/opt/graphite/bin/carbon-cache.py status > /dev/null 2>&1 ; echo $?"
if ReturnCode is equal to "1" then
        log "Starting Carbon"
        return true
end if
return false

Run the script in the Action.

  • Command snippet: /opt/graphite/bin/carbon-cache.py start

Step 3 - Install Grafana

  • brew install grafana
  • When it asks if you want to run it on startup, select y.
  • I think that's all I did. The rest of the configuration is done via UI. It runs HTTP on port 3000.

Step 4 - Install Indigraphs

  • Well, you're here aren't you? On your Indigo server:
mkdir -p ~/Documents/Indigo
cd ~/Documents/Indigo
git clone git@github.com:davidnewhall/indigraphs.git
  • You'll need to fix your PATH, like this:
  • export PATH=$PATH:/usr/local/bin:/Applications/Postgres.app/Contents/Versions/latest/bin:/Library/Application\ Support/Perceptive\ Automation/Indigo\ 7/IndigoPluginHost.app/Contents/MacOS/
  • I recommend putting this in your .bashrc or .profile - whatever works for you.
  • There is one dependency: sudo easy_install graphitesend
  • Test it: ./indigraphs.py
  • No output? It worked. Look in your Indigo Log. You should see something like Script [Indigraphs] Metrics Updated: 1342, Rows Skipped: 1272, Tables Scanned: 58.
  • Now to make it run every 5 minutes to auto-collect new data. Add another 5 minute schedule to Indigo, like this: Schedule for every 5 minutes.
  • Use "Execute Script" in the "Server Actions" like this: Execute Script.

Step 5 - Setup Grafana and add graphs.

  • Point your browser to port 3000 on whatever your host is that's running Indigo, possibly http://localhost:3000/
  • Add the graphite data source. Click the Grafana-logo menu in the top left, select Data Sources and click Add Data Source. Make it look like this:

Grafana Data Source

  • Setup your graphs. Grafana is a little intimidating at first, but it's not that bad. There's many ways to customize graphs and make it perfect.
  • Create a new Dashboard next. I named my House Report. In the dashboard, click Add Row and select Graph. Click on the name of the row (Panel Title) and then click Edit, like this:

Grafana Edit Row

  • From here, you can add metrics and customize the dashboard, like this:

Grafana Add Metric

  • Here's a partial screenshot of my system:

Grafana

About

Graphs for Indigo Data from SQL Logger Plugin

License:Other


Languages

Language:Python 97.1%Language:Shell 2.9%