cloudalchemy / ansible-grafana

Platform for analytics and monitoring

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

--skip-tags grafana_run still starts grafana due to untagged start task

kunickiaj opened this issue · comments

I was hoping to leverage the grafana_run tag to skip starting Grafana with a placeholder configuration similar to #151. In this case external database whose secrets have not been populated with real values, and I don't wish to attempt to connect to the database as this this playbook is run as part of an image build.

It seems that there's still an explicit start task that will attempt to (and fail) to start Grafana.

- name: Enable and start Grafana systemd unit
systemd:
name: grafana-server
enabled: true
state: started
daemon_reload: true

Can someone confirm that this task should be tagged with grafana_run? Or is there some other way to achieve completing this playbook with placeholder config values?

@bryanspears did you end up running into some issues with this despite adding the grafana_run tag?

@kunickiaj Hey! It's been a little while since I touched on this. I forked a while back and hadn't updated in some time, but it appears that configure restart task doesn't prevent --skip-tags grafana_run from doing what you're looking for. I also inject secrets at boot time vs. build (AMI) time.

For reference, but has the same restart stanza in the configure task.
https://github.com/rp-labs/ansible-grafana

Ah yeah I was hoping to avoid forking. In an effort to inject secrets at boot I was trying to use placeholder values.

I found that --skip-tags grafana_run seemed to work when using the sqlite3 db option however when I have an external database configured, the references "Enable and start Grafana systemd unit" would fail unless the database is actually reachable.

I was under the impression that this gets run because its not one of the tasks that's tagged.

  grafana: TASK [cloudalchemy.grafana : Enable and start Grafana systemd unit] ************
    grafana: fatal: [default]: FAILED! => {"changed": false, "msg": "Unable to start service grafana-server: Job for grafana-server.service failed because the control process exited with error code. See \"systemctl status grafana-server.service\" and \"journalctl -xe\" for details.\n"}
    grafana:
    grafana: RUNNING HANDLER [cloudalchemy.grafana : restart grafana] ***********************

Seems like you could add the grafana_run condition to that task. Unless grafana does some initializing on first start?

In order to avoid forking, I ended up (for now) modifying the /etc/sysconfig/grafana-server file loaded by the systemd unit with env vars using the GF_ syntax at boot to override the grafana.ini for properties that would prevent it from starting (and secrets).

That way I just leave grafana.ini minimally configured.

user-data script looks something like this:

#!/bin/bash -e

aws --region ${region} secretsmanager get-secret-value --secret-id ${secret_id} --query SecretString --output text | \
  jq -r 'to_entries|map("\(.key)=\(.value|tostring)")|.[]' >> /etc/sysconfig/grafana-server

systemctl daemon-reload
systemctl restart grafana-server

Solid option and I'd argue better than the route I took. Thanks, @kunickiaj

Arguably task

- name: Enable and start Grafana systemd unit
systemd:
name: grafana-server
enabled: true
state: started
daemon_reload: true
is not configuring grafana so I would say we should move the task out of configure.yml to main.yml and assign grafana_run tag to it. Care to create a PR?

Happy to open a PR. Do you think its reasonable for this task to enable the systemd unit (to start at boot) and daemon-reload without starting it?

Instead it could trigger the grafana restart handler (which is tagged grafana_run) (or a start handler to be a little more semantically accurate).

Do you think its reasonable for this task to enable the systemd unit (to start at boot) and daemon-reload without starting it?

SGTM

commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.