Is used to export one or more SQS queues metrics to Prometheus. This metrics can be used with k8s-prometheus-adapter to scaling pods k8s (HPA).
Read docs/k8s-prometheus-adapter.md for more informations to use with k8s-prometheus-adapter.
Currently exist some tools to getting SQS metrics, for example CloudWatch Exporter. But it has a long delay of 5~10 minutes to getting SQS metrics and using Tiamat you can this metrics in some seconds.
I use namespace monitoring
, but this is optional.
The following AWS policy is needed to use this program:
{
"Version": "2012-10-17",
"Id": "SQSTiamat",
"Statement": [
{
"Sid": "SQSTiamat",
"Effect": "Allow",
"Action": "sqs:GetQueueAttributes",
"Resource": "*"
}
]
}
If the program is running outside AWS and you don't have any configured credentials in the instance environment, you can set an AWS key and secret this way:
$ read awskey
<insert your key>
$ read awssecret
<insert your secret>
$ kubectl -n monitoring create secret generic aws-tiamat --from-literal=key=$awskey --from-literal=secret=$awssecret
If the program is running inside an instance that already has an IAM role attached and you don't want to use explicit credentials, set the key and secret as empty by running:
$ kubectl -n monitoring create secret generic aws-tiamat --from-literal=key="" --from-literal=secret=""
In configmap do you have set URLs Queues (SQS) and region.
Ex.: configmap_tiamat.yaml
apiVersion: v1
data:
config.json: |
{
"interval": 10,
"region": "sa-east-1",
"format_gauge_name": true,
"metric_type": "sqs",
"logs_enabled": true,
"queue_urls": [
"https://sqs.sa-east-1.amazonaws.com/123456789012/queue1",
"https://sqs.sa-east-1.amazonaws.com/123456789012/queue2"
]
}
kind: ConfigMap
metadata:
name: tiamat
namespace: monitoring
$ kubectl apply -f configmap_tiamat.yaml
interval
- Interval to getting queues metrics (in seconds).region
- Location SQS Region.format_gauge_name
- If istrue
the gauge names are reports in format:tiamat_<aws_account_id>_sqs_<queue_name>
. Default returntiamat
for all the gauge names. Note: The-
characters in queue names will be replaced by_
and the queues names are converted in lowercase.
Ex. format_gauge_name: true
:
.
.
.
# HELP tiamat_123456789012_sqs_queue1 Legacy metric, use total metrics instead
# TYPE tiamat_123456789012_sqs_queue1 gauge
tiamat_123456789012_sqs_queue1{metric_type="sqs",queue_account="123456789012",queue_name="queue1",queue_region="sa-east-1",queue_url="https://sqs.sa-east-1.amazonaws.com/739171219021/queue1"} 341547
# HELP tiamat_123456789012_sqs_queue1_total SQS Total Messages metrics
# TYPE tiamat_123456789012_sqs_queue1_total gauge
tiamat_123456789012_sqs_queue1_total{metric_type="sqs",queue_account="123456789012",queue_name="queue1",queue_region="sa-east-1",queue_url="https://sqs.sa-east-1.amazonaws.com/739171219021/queue1"} 341547
# HELP tiamat_123456789012_sqs_queue1_visible SQS Visible Messages metrics
# TYPE tiamat_123456789012_sqs_queue1_visible gauge
tiamat_123456789012_sqs_queue1_visible{metric_type="sqs",queue_account="123456789012",queue_name="queue1",queue_region="sa-east-1",queue_url="https://sqs.sa-east-1.amazonaws.com/739171219021/queue1"} 302473
# HELP tiamat_123456789012_sqs_queue1_in_flight SQS In Fight Messages metrics
# TYPE tiamat_123456789012_sqs_queue1_in_flight gauge
tiamat_123456789012_sqs_queue1_in_flight{metric_type="sqs",queue_account="123456789012",queue_name="queue1",queue_region="sa-east-1",queue_url="https://sqs.sa-east-1.amazonaws.com/739171219021/queue1"} 1253
metric_type
- In this moment is possible getting 'sqs' metrics, but in the future it will be possible to set others metrics (redis, dynamodb...).logs_enabled
- If istrue
logs are disabled. Default isfalse
.
deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
run: tiamat
release: prometheus-operator
name: tiamat
namespace: monitoring
spec:
replicas: 1
selector:
matchLabels:
run: tiamat
template:
metadata:
labels:
run: tiamat
spec:
containers:
- name: tiamat
image: hugobcar/tiamat:0.0.2
ports:
- containerPort: 5000
imagePullPolicy: Always
env:
- name: AWSKEY
valueFrom:
secretKeyRef:
name: aws-tiamat
key: key
- name: AWSSECRET
valueFrom:
secretKeyRef:
name: aws-tiamat
key: secret
- name: CONFIGMAP
value: tiamat
resources:
requests:
memory: "64Mi"
cpu: "100m"
limits:
memory: "128Mi"
cpu: "128m"
volumeMounts:
- name: config-volume
mountPath: /app/config.json
subPath: config.json
volumes:
- name: config-volume
configMap:
name: tiamat
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiamat
namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiamat-monitoring
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiamat
namespace: monitoring
$ kubectl apply -f deployment.yaml
service.yaml
apiVersion: v1
kind: Service
metadata:
labels:
release: prometheus-operator
run: tiamat
name: tiamat
namespace: monitoring
spec:
ports:
- name: tiamat
port: 5000
protocol: TCP
targetPort: 5000
selector:
run: tiamat
sessionAffinity: None
type: ClusterIP
$ kubectl apply -f service.yaml