Giters

Yleisradio / aws-maven

Fork to add support for assumed roles

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Profile Support Not Working

mjallday opened this issue · comments

commented

Using the AWS credentials directly works

AWS_ACCESS_KEY_ID=XXX AWS_SECRET_ACCESS_KEY=YYY mvn -f site-pom.xml deploy

 
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Very Good Parent POM 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ vg-parent-pom ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/marshall/code/vg/commons-data/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ vg-parent-pom ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ vg-parent-pom ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/marshall/code/vg/commons-data/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ vg-parent-pom ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ vg-parent-pom ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ vg-parent-pom ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ vg-parent-pom ---
[INFO] Installing /Users/marshall/code/vg/commons-data/target/vg-parent-pom-0.0.1-SNAPSHOT.jar to /Users/marshall/.m2/repository/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-SNAPSHOT.jar
[INFO] Installing /Users/marshall/code/vg/commons-data/site-pom.xml to /Users/marshall/.m2/repository/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-SNAPSHOT.pom
[INFO]
[INFO] --- maven-deploy-plugin:2.7:deploy (default-deploy) @ vg-parent-pom ---
Downloading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/maven-metadata.xml
Uploading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-20160714.234542-1.jar
Uploaded: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-20160714.234542-1.jar (4 KB at 2.4 KB/sec)
Uploading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-20160714.234542-1.pom
Uploaded: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-20160714.234542-1.pom (11 KB at 7.6 KB/sec)
Downloading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/maven-metadata.xml
Uploading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/maven-metadata.xml
Uploaded: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/maven-metadata.xml (775 B at 0.5 KB/sec)
Uploading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/maven-metadata.xml
Uploaded: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/maven-metadata.xml (285 B at 0.3 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.683 s
[INFO] Finished at: 2016-07-14T16:45:47-07:00
[INFO] Final Memory: 19M/304M
[INFO] ------------------------------------------------------------------------

Using AWS CLI with the profile works

AWS_PROFILE=vg aws s3 ls s3://very-good/software/snapshot/
                           PRE com/
2016-07-14 16:45:48          0

Using the profile does not work

AWS_PROFILE=vg mvn -f site-pom.xml deploy

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Very Good Parent POM 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ vg-parent-pom ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/marshall/code/vg/commons-data/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ vg-parent-pom ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ vg-parent-pom ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/marshall/code/vg/commons-data/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ vg-parent-pom ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ vg-parent-pom ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ vg-parent-pom ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ vg-parent-pom ---
[INFO] Installing /Users/marshall/code/vg/commons-data/target/vg-parent-pom-0.0.1-SNAPSHOT.jar to /Users/marshall/.m2/repository/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-SNAPSHOT.jar
[INFO] Installing /Users/marshall/code/vg/commons-data/site-pom.xml to /Users/marshall/.m2/repository/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-SNAPSHOT.pom
[INFO]
[INFO] --- maven-deploy-plugin:2.7:deploy (default-deploy) @ vg-parent-pom ---
Downloading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/maven-metadata.xml
[INFO] Skip unsupported property name role_arn in profile [vault-prod].
[INFO] Skip unsupported property name source_profile in profile [vault-prod].
[INFO] Skip unsupported property name mfa_serial in profile [vault-prod].
[WARNING] Could not transfer metadata com.verygood:vg-parent-pom:0.0.1-SNAPSHOT/maven-metadata.xml from/to vg-snapshot (s3://very-good/software/snapshot/): Unable to load AWS credentials from any provider in the chain
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.897 s
[INFO] Finished at: 2016-07-14T16:50:33-07:00
[INFO] Final Memory: 15M/245M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project vg-parent-pom: Failed to retrieve remote metadata com.verygood:vg-parent-pom:0.0.1-SNAPSHOT/maven-metadata.xml: Could not transfer metadata com.verygood:vg-parent-pom:0.0.1-SNAPSHOT/maven-metadata.xml from/to vg-snapshot (s3://very-good/software/snapshot/): Unable to load AWS credentials from any provider in the chain -> [Help 1]
[ERROR]

Extension is installed via

        <extensions>
            <extension>
                <groupId>fi.yle.tools</groupId>
                <artifactId>aws-maven</artifactId>
                <version>1.4.0</version>
            </extension>
        </extensions>

and

    <pluginRepositories>
        <pluginRepository>
            <id>yle-public</id>
            <name>Yle public repository</name>
            <url>http://maven.c4.yle.fi/release</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </pluginRepository>
    </pluginRepositories>

Any ideas here?

commented

Update - Turns out I have some profiles in ~/.aws/credentials which use IAM based cross account signin and this version of the AWS SDK does not handle this well.

You can see this in the above output where it gives the message [INFO] Skip unsupported property name role_arn in profile [vault-prod].. Upgrading AWS SDK should resolve that. Any chance you guys will do that? Feel free to close this issue if not.

commented

It is currently vacation season here, but will check this in few weeks - when back at the office, unless Matti has spare time already.

commented

@mjallday I'm not sure if I understood the big picture, so sorry about this recap. So if you try directly by using access key and secret it works but if you use profile (which contains the key and secret) it doesn't, right?

Hmmm... I'm using this tool daily with AWS_PROFILE and it's working fine. What kind of error message do you get when you try to run deploy with profile? Also, what do you mean with that "IAM based cross account signin"?

commented

@milankinen If I understand the last comment by @mjallday correctly, he is trying to use a profile that does not contain aws_access_key_id and aws_secret_access_key entries, but role_arn and source_profile entries instead, i.e. a profile entry to assume a role (based on the credentials of the source_profile).

Support for this type of profile got added to the AWS Java SDK with release 1.10.63:

Support for using assume roles in the shared credentials file much like the CLI.

As this project is currently using the AWS Java SDK version 1.10.59, updating that should automatically add the missing support for the new profile type.