With sso-session config "Required key: :secret_access_key must be a string, but instead is..."

[chrismo]

Environment

• Elixir & Erlang versions (elixir --version):

Erlang/OTP 25 [erts-13.1.3] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]
Elixir 1.14.2 (compiled with Erlang/OTP 25)

• ExAws version mix deps |grep ex_aws

* ex_aws 2.4.2 (Hex package) (mix)
  locked at 2.4.2 (ex_aws) 0a2c07bd
* ex_aws_s3 2.4.0 (Hex package) (mix)
  locked at 2.4.0 (ex_aws_s3) 85dda6e2

• HTTP client version. IE for hackney do mix deps | grep hackney

* hackney 1.18.1 (Hex package) (rebar3)
  locked at 1.18.1 (hackney) a4ecdaff

Current behavior

Using the newer sso-session config which supports automatic token refresh.

~/.aws/config

[profile dev]
sso_session = dscout
sso_account_id = xxxxxxxx
sso_role_name = StuffAccess
region = us-east-2
output = json

[sso-session dscout]
sso_start_url = https://xxxxxxxx.awsapps.com/start#/
sso_region = us-east-2
sso_registration_scopes = sso:account:access

runtime.exs

config :ex_aws,
  access_key_id: [
    {:system, "AWS_ACCESS_KEY_ID"},
    {:awscli, System.get_env("AWS_PROFILE", "dev"), 30},
    :instance_role
  ],
  secret_access_key: [
    {:system, "AWS_SECRET_ACCESS_KEY"},
    {:awscli, System.get_env("AWS_PROFILE", "dev"), 30},
    :instance_role
  ]

iex

iex(3)> ...
iex(4)> ExAws.request(ExAws.S3.head_object(bucket, key))
{:error,
 "Required key: :secret_access_key must be a string, but instead is [{:system, \"AWS_SECRET_ACCESS_KEY\"}, {:awscli, \"dev\", 30}, :instance_role]"}

Expected behavior

Expected behavior is that the request would succeed. If the configuration changes back to the legacy config, then the request works.

~/.aws/config

[profile dev]
sso_start_url = https://xxxxxxxxx.awsapps.com/start#/
sso_region = us-east-2
sso_account_id = xxxxxxxx
sso_role_name = StuffAccess
region = us-east-2
output = json

iex

iex(5)> ExAws.request(ExAws.S3.head_object(bucket, key))
{:ok, ...

[chrismo]

fwiw, terraform has a similar limitation: hashicorp/terraform-provider-aws#28263

[Bernard Duggan]

Okay, took me a while to pin down, because that's a super misleading error message (and I'm just adding some code to improve it now). But yes, the upshot is that nobody has written code to support that style of auth config. PRs welcome :)

Thanks for the detailed report.

[Bernard Duggan]

Incidentally, if you're looking to add support, the place to do it is lib/ex_aws/credentials_ini/file.ex :)

[Yu Matsuzawa]

I can confirm, with {:awscli, "default", 30} it works but it does not for profiles other than "default"
A bit looked into the code but could not track down the cause. Strangely credentials_ini/file.ex DOES have profile resolution mechanism implemented so there must be wiring issues somewhere.

ex_aws/lib/ex_aws/credentials_ini/file.ex

Lines 282 to 292 in 29c0510

 defp profile_from_name(:system) do 
   profile_name_from_env() 
   |> profile_from_name() 
 end 
  
 defp profile_from_name("default"), do: "default" 
 defp profile_from_name(other), do: "profile #{other}" 
  
 defp profile_name_from_env() do 
   System.get_env("AWS_PROFILE") || "default" 
 end 

[Yu Matsuzawa]

Ah my previous comment was somewhat off. As @chrismo wrote,

Using the newer sso-session config which supports automatic token refresh.

Expected behavior is that the request would succeed. If the configuration changes back to the legacy config, then the request works.

So I can confirm that the problem is, ex_aws's awscli config provider not yet supporting newer aws sso config structure (sso-session).

[Yu Matsuzawa]

For the time being, I'm going to revert my sso config to legacy structure as a workaround.