Add a k/v tree mechanism to switch_monitor
sepulworld opened this issue · comments
per discussion with @hoadley -
would be great to reduce the number of switch_monitors that need to run by allowing the switch_monitor to traverse a key/value tree. Along with the tree traversal, we need to add more information to the key being set. We need to set a JSON object in the key that looks like this
{
"epoch": 1492482679470,
"frequency": 3000
}
Let's say we have have 3 distinct periodic services "SilverBullet", "GreyBullet", "GoldBullet"
Here are the target keys to be updated and monitored
deadman/DeadmanSetSilverBulletPeriodicProcess
deadman/DeadmanSetGreyBulletPeriodicProcess
deadman/DeadmanSetGoldBulletPeriodicProcess
Each of these could contain their own unique "frequency" settings.
We could spawn one switch_monitor to monitor the deadman/ k/v tree like so:
job "DeadmanMonitoring" {
datacenters = ["dc1"]
type = "service"
group "monitor" {
task "DeadmanMonitorSilverBulletPeriodicProcess" {
driver = "docker"
config {
image = "sepulworld/deadman-check"
command = "switch_monitor"
args = [
"--host",
"10.0.0.1",
"--port",
"8500",
"--key-tree",
"deadman/",
"--alert-to",
"slackroom",
"--daemon",
"--daemon-sleep",
"900"]
}
resources {
cpu = 100
memory = 256
}
env {
SLACK_API_TOKEN = "YourSlackApiToken"
}
}
}
}
This may be backwards incompatible, but perhaps we can make the changes to work along side current behavior.