sverma / httpJsonStats

Script for collecting stats via json and sending to graphite

httpJsonStats

httpJsonStats is a script to parse key/value pairs out of Json output from a web url and format it for the Graphite input.

Example Lets say we have URL which outputs a json format stats for a application

# curl http://webserver1.domain.com:9090/context/stats.sessions.all
{"running":31,"total":1683,"total_time":14243815942,"average_time":"02:23:42"}

similarly We can have other servers also which output in similar format

It accepts a json config to form proper urls , parse them , construct key value pairs and send it to Graphite server , Example Config is Below

{
  "global": { 
    "GRAPHITE_SERVER" : "YOUR_GRAPHITE_SERVER", 
    "GRAPHITE_PORT"   : 2003,
    "INTERVAL"      : 20, 
        "LOG_FILE"       : "/tmp/httpJsonStats.log", 
        "ERR_LOG_FILE"   : "/tmp/httpJsonStats.log", 
        "PID_FILE"       : "/tmp/httpJsonStats.pid"
  }, 
 "chirkut": 
  {
      "host": "server1.some.com,server2.some.com",
      "port": "9090",
      "groups": {
          "packet_stats": {
              "URN": "/context/stats.all"
          },
          "Stream_Aggregated_Stats": {
              "URN": "/context/stats.streams.all"
          },
          "Stream_Aggregated_Sessions": {
              "URN": "/context/stats.sessions.all"
          }

      }
  }

}

Above config will Form URLs as below

server1.some.com:9090/context/stats.all

server2.some.com:9090/context/stats.all

and Sends stats to graphite according to key/value pairs output by above URLs.

The Graphite naming scheme it create is as follows

chirkut.packet_stats.<HOSTNAME>.

You can filter particular keys from a json response . "filter" can be a regular expression Example :

   "NonHeapMemoryUsageUsed": { 
        "URN": "/jolokia/read/java.lang:type=Memory/NonHeapMemoryUsage/used" , 
        "filter": "value"
    } , 

You can also override port config for a particular group . Example:

    "connectedEntities": {
      "URN": "/stats/connectedEntities.json", 
      "port" : "8080"
    } , 

You can view a screenshot for further clarification , Installation

TODO