Feature: `aws-vault export --credentials`

Question

Feature: `aws-vault export --credentials`

mtibben opened this issue a year ago · comments

I wonder if we should create a aws-vault export --stored-credentials PROFILE , a new flag that will only output the master credentials in the vault and never attempt to create temporary credentials.

This seems to be the use-case of #1176 and #1180 which have been raised after the introduction of credential_process as a way to source credentials in v7.

Would this help avoid creating extra profiles just for credentials?

Thoughts @ArjunDandagi @dgholz @Ketouem @jweyrich @olfway

Michael Tibben · Answer 1 · Tue Mar 07 2023 08:57:32 GMT+0800 (China Standard Time)

hmmm actually not sure that works

olfway · Answer 2 · Tue Mar 07 2023 15:16:59 GMT+0800 (China Standard Time)

May be the old command "aws-vault exec --json" could be modified to ignore credential_process - so the backward compatibility would be restored?

I understand that it's deprecated, but it will give us a time to adopt our configs, upgrade aws-vault, etc

olfway · Answer 3 · Tue Mar 07 2023 15:29:21 GMT+0800 (China Standard Time)

I wonder if we should create a aws-vault export --stored-credentials PROFILE , a new flag that will only output the master credentials in the vault and never attempt to create temporary credentials.

In my case I do want aws-vault to ask for mfa and create temporary credentials
I think the flag should be something like aws-vault export --no-credential-process

Or another way could be to add environment variables to the new credential process like AWS_VAULT_CONFIG_FILE, AWS_VAULT_PROFILE_NAME, AWS_VAULT_CREDENTIAL_PROCESS and then avoid going into infinite credential_process loop based on those variables

Michael Tibben · Answer 4 · Tue Mar 07 2023 15:56:41 GMT+0800 (China Standard Time)

I think the flag should be something like aws-vault export --no-credential-process

Yeah it's a decent idea

Daniel George Holz · Answer 5 · Tue Mar 07 2023 17:34:58 GMT+0800 (China Standard Time)

That sounds good, but that's also what I thought --no-session did. Obviously it's still following credential_process, I reckon it should not (and be deprecated in favour of whichever suggestion ends up being selected from this issue).

Michael Tibben · Answer 6 · Tue Mar 07 2023 20:33:44 GMT+0800 (China Standard Time)

--no-session specifically disables STS GetSessionToken which aws-vault normally utilises to share MFA creds between profiles. So that's a bit different.

The use-case here is using aws-vault as a cache, which requires aws-vault to ignore the credential_process, but use the rest of the profile config when creating the credentials.

So I'm thinking 2 new flags for export

--ignore-credential-process to ignore credential_process

--credential-process-cache an alias for --format=json --ignore-credential-process

Any feedback on that?

olfway · Answer 7 · Tue Mar 07 2023 21:53:11 GMT+0800 (China Standard Time)

For me it seems enough to have only one flag --ignore-credential-process / --no-credential-process

The --credential-process-cache looks not obvious to me and I guess it will be always used in config, so it's not too difficult to write two flags instead of this one

olfway · Answer 8 · Tue Mar 07 2023 21:55:34 GMT+0800 (China Standard Time)

But please, consider changing aws-vault exec --json to be an alias for aws-vault export --ignore-credential-process --format=json

This will restore backward compatibility and will give us a time to upgrade

Jardel Weyrich · Answer 9 · Tue Mar 07 2023 22:33:01 GMT+0800 (China Standard Time)

I'm not sure I followed your idea. But I'll try to explain my scenario:

This is my $HOME/.aws/config:

[profile company-prod]
region=sa-east-1
credential_process=aws-vault export --no-session --format=json company-prod

The profile was configured via aws-vault add company-prod. I use the macOS keychain.

I used to run aws --profile=company-prod s3 ls, but since v7 this no longer works.
I was originally using credential_process=aws-vault exec company-prod --json, then I updated the command to use export hoping it would fix it for v7.
Am I doing something wrong?

I'm running macOS 13.1, and this is the --debug output:

2023/03/07 11:34:30 aws-vault v7.0.1
2023/03/07 11:34:30 Using prompt driver: terminal
2023/03/07 11:34:30 Loading config file /Users/myuser/.aws/config
2023/03/07 11:34:30 Parsing config file /Users/myuser/.aws/config
2023/03/07 11:34:30 [keyring] Considering backends: [keychain]
2023/03/07 11:34:30 profile company-prod: using credential process
2023/03/07 11:34:30 [keyring] Querying keychain for service="aws-vault", keychain="aws-vault.keychain"
2023/03/07 11:34:30 [keyring] Found 6 results
2023/03/07 11:34:30 [keyring] Querying keychain for service="aws-vault", keychain="aws-vault.keychain"
2023/03/07 11:34:30 [keyring] Found 6 results
2023/03/07 11:34:30 [keyring] Querying keychain for service="aws-vault", account="credential_process,REDACTED,,-62135596800", keychain="aws-vault.keychain"
2023/03/07 11:34:30 [keyring] No results found
<... wait for various minutes>
aws-vault: error: exec: Failed to get credentials for company-prod: running command "aws-vault export --no-session --format=json company-prod": exit status 1

Daniel George Holz · Answer 10 · Tue Mar 07 2023 23:07:55 GMT+0800 (China Standard Time)

--no-session specifically disables STS GetSessionToken which aws-vault normally utilises to share MFA creds between profiles. So that's a bit different.

I don't see the difference: skipping the STS calls means the original AWS credentials are used. And for MFA creds, I think this means the actual AWS credentials that may be protected with MFA? I think that matches what you proposed in the original post.

edit: I read the code and now it's clear to me what you meant.

I still think, whatever solution you decide on, that --no-session should also behave similarly (to ignore credential_process in the AWS config). That's to support using aws-vault from credential_process & avoid it calling itself.

edit: thinking about it more, this would break @jmczerk's use-case. I guess there needs to be a option that can be passed to aws-vault for when it's being configured to be called as a credential_process. I like @olfway's suggested --no-credential-process for that!

The use-case here is using aws-vault as a cache

This is the new behaviour from #1087? I feel like I want aws-vault to handle securely accessing my AWS credentials, assuming roles from profiles, and caching the session credentials. I see how the last part could be used for caching non-session credentials from external processes, but I think that makes more sense as an extension of aws-vault add.

John Czerkowicz · Answer 11 · Tue Mar 07 2023 23:48:19 GMT+0800 (China Standard Time)

Apologies for introducing this issue.

Given my use case seems to be less common than those impacted, would it make sense to take the approach of #1117 where the the aws-vault external credential provider uses an aws-vault-specific config option (i.e. aws_vault_credential_process in the referenced PR) rather than shadowing the AWS CLI's credential_process directly?

Michael Tibben · Answer 12 · Wed Mar 08 2023 10:06:49 GMT+0800 (China Standard Time)

The core problem here is that there are a few different modes aws-vault is used in. And unfortunately many configs have been set up ambiguously by specifying multiple credential sources.

I have collected the use-cases that I understand aws-vault is being used in.

Use-case 1: aws-vault is the executor and provides the environment

One group of people (myself and my team included) uses aws-vault exclusively as a runner, where aws-vault provides the environment and runs a command.

[profile my_profile_master]
# master credentials stored in aws-vault

[profile my_profile_role]
source_profile=my_profile_master
role_arn=xxx

aws-vault exec my_profile_master ./my-command   # success, uses sts session generated by aws-vault
aws-vault exec my_profile_role ./my-command     # success, uses role creds generated by aws-vault

AWS_PROFILE=my_profile_master ./my-command      # Not expected to be functional
AWS_PROFILE=my_profile_role ./my-command        # Not expected to be functional

In this scenario, the profile name and aws config is used exclusively by aws-vault, which provides the environment for the command to run in.

This is a very unix-y and 12-factor approach. It's the original (and I consider the primary) use-case of aws-vault - it's why its aws-vault exec exists.

Use-case 2: aws-vault is a "master credentials vault" for AWS SDK

A second group of people use aws-vault in credential_process to provide master creds in the config. This is more in-line with the AWS SDK way of approaching the problem via credential_process and AWS_PROFILE

[profile my_profile_master]
credential_process = aws-vault6 exec --json --no-session my_profile_master  # aws-vault7 export --format=json --no-session my_profile_master

[profile my_profile_role]
source_profile=my_profile_master
role_arn=xxx

aws-vault exec my_profile_master ./my-command   # Not expected to be functional
aws-vault exec my_profile_role ./my-command     # Not expected to be functional

AWS_PROFILE=my_profile_master ./my-command      # v6: success (uses aws-vault master creds)
                                                # v7: broken
AWS_PROFILE==my_profile_role ./my-command       # v6: success (uses aws-vault master creds + SDK role)
                                                # v7: broken

The problem here is that my_profile_master profile is used in 2 different contexts

AWS SDK uses it to execute credential_process
aws-vault uses it to work out how to create credentials (and now in v7 executes credential_process as part of that)

Use-case 3: aws-vault is a "MFA session cache" for AWS SDK

Very similar to Use-case 2, another group of people want aws-vault to cache STS MFA credentials between profiles. This means they are not forced to re-authenticate with MFA every time they switch profiles.

[profile my_profile_master]
mfa_serial=mmm
credential_process = aws-vault6 exec --json my_profile_master  # aws-vault7 export --format=json my_profile_master

[profile my_profile_role]
source_profile=my_profile_master
mfa_serial=mmm
role_arn=xxx1

[profile my_profile_role2]
source_profile=my_profile_master
mfa_serial=mmm
role_arn=xxx2

aws-vault exec my_profile_master ./my-command   # Not expected to be functional
aws-vault exec my_profile_role ./my-command     # Not expected to be functional

AWS_PROFILE=my_profile_master ./my-command      # v6: success (uses aws-vault session)
                                                # v7: broken
AWS_PROFILE=my_profile_role ./my-command        # v6: success (uses aws-vault session + SDK role)
                                                # v7: broken

This config has the same problem as Use-case 2, but there is also an additional point of confusion. The confusion here is that the user has set master creds via aws-vault add my_profile_master, but expecting credential_process = aws-vault export my_profile_master to return a session token. This requires aws-vault to have access to the config file in order to read config like the mfa_serial.

Use-case 4: aws-vault caches alternative credential sources like sso_start_url, etc

aws-vault caches credentials from alternative credential sources like sso_start_url, web_identity_token_process. In #1087, this was extended to support credential_process as well.

[profile my_profile_using_sso]
sso_start_url = https://mycompany.awsapps.com/start

[profile my_profile_using_process]
credential_process = my-custom-creds-cmd

aws-vault exec my_profile_using_sso ./my-command       # success, uses aws-vault caching
aws-vault exec my_profile_using_process ./my-command   # success, uses aws-vault caching

AWS_PROFILE=my_profile_using_sso ./my-command          # success, no caching
AWS_PROFILE=my_profile_using_process ./my-command      # success, no caching

Solutions

There are a few possible solutions to the problems identified in use-case 2 and 3.

Solution 1. Use a separate profile name for the master creds

Use-case 2:

[profile my_profile_vault]
# master credentials stored via 'aws-vault add my_profile_vault'

[profile my_profile_master]
# get master credentials
credential_process = aws-vault7 export --format=json --no-session my_profile_vault

[profile my_profile_role]
# master creds sourced, role assumed
source_profile=my_profile_master
role_arn=xxx
mfa_serial=mmm

Use-case 3:

[profile my_profile_vault]
# master credentials stored via 'aws-vault add my_profile_vault'
mfa_serial=mmm

[profile my_profile_session]
# get STS session
credential_process = aws-vault7 export --format=json my_profile_vault
mfa_serial=mmm

[profile my_profile_role]
# session sourced, role assumed
source_profile=my_profile_session
role_arn=xxx
mfa_serial=mmm

Pros:

This solution is functional right now

Cons:

requires a separate profile name for the master creds, so some folks need to update their configs. But that's why aws-vault v7 is a major version bump.

Solution 2. Tell aws-vault to output master credentials only with `--credentials`

[profile my_profile_master]
mfa_serial=mmm
credential_process = aws-vault7 export --credentials --format=json my_profile_master

Cons:

This addresses use-case 2, but doesn't address use-case 3
Confusing when used in conjunction with --no-session

Solution 3. Tell aws-vault to not load config with `--no-config`

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --no-config --format=json my_profile

This is an interesting idea because it cuts through to the root cause of the problem - both AWS SDK and aws-vault are using the config file. If the config file is actually not required by aws-vault, then it can be ignored completely. But then if it is required, we're back to square 1

Pros:

Simple to understand - config files are not loaded at all, only the vaulting and STS session functionality of aws-vault is used

Cons:

MFA config won't be loaded for use-case 3
What other use-cases would this break?

Solution 4. Tell aws-vault to ignore `credential_process` in the active profile with `--ignore-credential-process`

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --ignore-credential-process --format=json my_profile

Pros:

surgically targets the problem

Cons:

feels arbitrary to have a flag that only applies to one config in one profile

Solution 5. Tell aws-vault to disable `credential_process` in all profiles with `--disable-credential-process`

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --disable-credential-process --format=json my_profile

Pros:

generally targets the problem

Cons:

untargeted, i'm sure someone will come up with a use-case where this is not desirable

Solution 6. Change `--no-session` to mean "only return master credentials" instead of "don't use STS sessions"

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --no-session --format=json my_profile

Cons:

This addresses use-case 2, but doesn't address use-case 3

Solution 7. Add a `aws_vault_ignore_credential_process` to config

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --format=json my_profile
aws_vault_ignore_credential_process=true

Pros:

gives aws-vault the context to ignore the credential_process in the active profile

Cons:

some users need to update their configs (but that's why it's a major version bump)

Solution 8. aws-vault ignores `credential_process` and only recognises `aws_vault_credential_process`

[profile my_profile]
mfa_serial=mmm
credential_process = aws-vault7 export --format=json my_profile

[profile my_profile_using_process]
aws_vault_credential_process = my-custom-creds-cmd

Pros:

keeps existing configs working

Cons:

breaks ability to execute AWS_PROFILE=my_profile_using_sso ./my-command in Use-case 4
aws-vault doesn't support credential_process

Solution 9. Preference stored credentials over other credential sources

As suggested by @dgholz, aws-vault could prefer stored credentials over other credential sources. This would mean that if you have a profile with both stored credentials and credential_process set, aws-vault would use the stored credentials instead of executing the credential_process.

We would likely want to log the conflict and the fact that the stored credentials are being used.

Pros

keeps existing configs working
matches AWS SDK behaviour (which prefers the shared credentials file over credential_process)

Over to you. Have I missed any Use-cases? Are there any other solutions? What do you think of my commentary and the solutions I've proposed?

olfway · Answer 13 · Wed Mar 08 2023 16:46:11 GMT+0800 (China Standard Time)

I'm using aws-vault similar to case 3 (mfa session cache) and actually v6 works for both aws-vault exec profile ./mycmd.sh and AWS_PROFILE=profile ./mycmd.sh

I vote for solution 4 (--ignore-credential-process) - it fixes problem, will allow to use aws-vault exec or mycmd directly and no need to add extra profiles

I prefer it more than solution 7 (aws_vault_ignore_credential_process=true) because command line flag is more flexible – I can add it to credential_process or use it while running aws-vault export directly from my shell or some other script

Cyril Thomas · Answer 14 · Wed Mar 08 2023 17:08:27 GMT+0800 (China Standard Time)

Regardless of the future chose solution, please note that this new flag should be documented in (https://github.com/99designs/aws-vault/blob/master/USAGE.md#using-credential_process)[the USAGE documentation]. I've stumbled upon the "initial bug" by actually following it and others may fall into the same trap as I did.

Daniel George Holz · Answer 15 · Wed Mar 08 2023 21:54:56 GMT+0800 (China Standard Time)

@mtibben thanks for gathering these use cases. I actually have another use case: combined 1 & 2 & 3:

aws-vault exec my_profile_master ./my-command   # success, uses sts session generated by aws-vault
aws-vault exec my_profile_role ./my-command     # success, uses role creds generated by aws-vault

AWS_PROFILE=my_profile_master ./my-command      # success (uses sts session generated by aws-vault)
AWS_PROFILE=my_profile_role ./my-command        # success (uses sts session generated by aws-vault + SDK role)

which worked fine in v6, and means I get prompted once for my MFA token while the session is still active, regardless of which tool I use.

My proposal for a solution is: don't call credential_process if there's stored credentials that match the profile. I've hacked it together in dgholz#1. Do you think this is worth continuing?

Michael Tibben · Answer 16 · Thu Mar 09 2023 14:21:29 GMT+0800 (China Standard Time)

another use case: combined 1 & 2 & 3

Sorry @dgholz I don't exactly understand what that means. Can you provide a simple-as-possible example of the config, commands, and expected behaviour?

My proposal for a solution is: don't call credential_process if there's stored credentials that match the profile. I've hacked it together in dgholz#1. Do you think this is worth continuing?

🤔 Simple and effective. It might just work. Great idea @dgholz. I'll add it to the list

Michael Tibben · Answer 17 · Thu Mar 09 2023 18:13:02 GMT+0800 (China Standard Time)

A pre-release fix has been released v7.0.2-beta2 release with the Solution 9 fix (#1183) suggested by @dgholz.

Please test it and provide any feedback to ensure that this is doing the right thing

Michael Tibben · Answer 18 · Fri Mar 10 2023 04:52:01 GMT+0800 (China Standard Time)

Fixed in #1183

Feature: `aws-vault export --credentials`

Use-case 1: aws-vault is the executor and provides the environment

Use-case 2: aws-vault is a "master credentials vault" for AWS SDK

Use-case 3: aws-vault is a "MFA session cache" for AWS SDK

Use-case 4: aws-vault caches alternative credential sources like sso_start_url, etc

Solutions

Solution 1. Use a separate profile name for the master creds

Solution 2. Tell aws-vault to output master credentials only with --credentials

Solution 3. Tell aws-vault to not load config with --no-config

Solution 4. Tell aws-vault to ignore credential_process in the active profile with --ignore-credential-process

Solution 5. Tell aws-vault to disable credential_process in all profiles with --disable-credential-process

Solution 6. Change --no-session to mean "only return master credentials" instead of "don't use STS sessions"

Solution 7. Add a aws_vault_ignore_credential_process to config

Solution 8. aws-vault ignores credential_process and only recognises aws_vault_credential_process

Solution 9. Preference stored credentials over other credential sources

Solution 2. Tell aws-vault to output master credentials only with `--credentials`

Solution 3. Tell aws-vault to not load config with `--no-config`

Solution 4. Tell aws-vault to ignore `credential_process` in the active profile with `--ignore-credential-process`

Solution 5. Tell aws-vault to disable `credential_process` in all profiles with `--disable-credential-process`

Solution 6. Change `--no-session` to mean "only return master credentials" instead of "don't use STS sessions"

Solution 7. Add a `aws_vault_ignore_credential_process` to config

Solution 8. aws-vault ignores `credential_process` and only recognises `aws_vault_credential_process`