Some help needed

[Frode Egeland]

Hi again,

While I was developing it, my little "secretupdater" app that receives webhooks and pulls secrets from Confidant and pushes Secrets into kubernetes was working well.
Now that I have SAML authentication, rather than the header "authentication", I'm having no end of confusion and trouble.

How I've built it:
I'm using the python confidant client as a library, filling in the config this way, with the values pulled from env variables:

client = ConfidantClient(
            url=app.config.get('CONFIDANT_SERVER_URL'),
            auth_key=app.config.get('CONFIDANT_SERVER_AUTH_KEY'),
            auth_context=app.config.get('CONFIDANT_SERVER_AUTH_CONTEXT'),
            assume_role=app.config.get('CONFIDANT_SERVER_IAM_ROLE'),
            region=app.config.get('CONFIDANT_SERVER_AWS_REGION'),
            token_cache_file=''  # Not caching the token
        )

On receiving the webhook from Confidant, the secretupdater extracts the affected services, then goes through and pulls all the credentials for each affected service, with:

for service in affected_services:
    service_details = client.get_service(service=service)

(At least, it would, if it worked.)

Then it converts them to k8s Secret format and pushes those into k8s.

I'm now managed to confuse myself (and my colleague) about what exactly is happening when the confidant client talks to confidant.
It seems like I've got some problem involving IAM, KMS and roles interacting with grants, but I don't really follow what's happening.
Assuming we have a Confidant server running on an instance that has an IAM instance role, and we want a client, running on the same instance to be able to access Confidant credentials as mentioned above, would you mind describing the flow of what is happening, what options I need to use to access all services' credentials from this client, and what the involvement with KMS Grants and IAM Roles are?

I'd really appreciate any light you can shed on this - I've been trying all sorts of permutations on the config options, and I keep getting errors like:

ERROR:root:Failed to validate token.
Traceback (most recent call last):
  File "/srv/confidant/confidant/keymanager.py", line 146, in decrypt_token
    EncryptionContext=context
  File "/usr/local/lib/python2.7/site-packages/botocore/client.py", line 159, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python2.7/site-packages/botocore/client.py", line 494, in _make_api_call
    raise ClientError(parsed_response, operation_name)
ClientError: An error occurred (ValidationException) when calling the Decrypt operation: Invalid EncryptionContext
ERROR:root:Failed to decrypt authentication token.
Traceback (most recent call last):
  File "/srv/confidant/confidant/authnz/__init__.py", line 176, in decorated
    kms_auth_data['token']
  File "/srv/confidant/confidant/keymanager.py", line 176, in decrypt_token
    raise TokenDecryptionError('Authentication error. General error.')
TokenDecryptionError: Authentication error. General error.
WARNING:root:Access denied for dummy. Authentication Failed.
10.44.13.129 - - [20/Feb/2017:04:29:12 +0000] "GET /v1/services/dummy HTTP/1.1" 403 234 "-" "python-requests/2.13.0"

(dummy is, in this instance, the name of a service in Confidant, listed in the context's "from".)

I guess my first question is: is my intended workflow possible?

[Ryan Lane]

Your workflow is possible, but if you have a single IAM role that needs to access multiple services, you need to do one of two things:

1. Make roles for each service, allow your instance role to assume these roles, then initialize a confidant client for each service, passing in the assumed role.
2. Use IAM policy to allow the instance to generate authentication tokens for any of the services. Initialize a confidant client for each service you want to fetch, adjusting the auth context for each intialization.

I'd recommend 2, especially since your IAM policy may already allow this for the key. Something to note here is that the auth context you use for the client needs to be scoped to the service you're fetching; so for instance, you probably want a context like:

{'to': 'confidant-production', 'from': 'serviceA-production', 'user_type': 'service'}

I think here you also don't need to assume any role, since you're already using the confidant server's IAM role.

[Frode Egeland]

Thanks for replying. 😃

Yes, 2 sounds like it would be the best fit, indeed. And not assuming a role sounds like it would simplify things a bit, too.
OK, I'll dig through the documentation to see how to implement the appropriate IAM policy.

In regards to the context scoping, the "from": "serviceA-production" value would be the name of the Service as created in confidant, right?
And what determines the value of the to in this case?

[Ryan Lane]

Yep, the from is the service name. The to is the auth context of confidant server that you have set in the environment variable.

We have some docs on setting up kmsauth with IAM policy:

https://lyft.github.io/confidant/advanced/kms_auth/

[Frode Egeland]

I got it working as per suggestion 2 😀

Thanks for the help!

[Ryan Lane]

Awesome. Glad to see it's working!