gh gei migrate-repo generates invalid pre-signed urls for s3 buckets
brokenjacobs opened this issue · comments
This on GitHub Enterprise Server version 3.10.2 gh cli v1.4.0
My migrations run and upload files to my S3 bucket, but the generated urls have invalid signatures:
[2023-11-17 13:13:00] [INFO] Waiting for archive with id 3 generation to finish. Current status: exported
[2023-11-17 13:13:01] [INFO] Archive (git) download url: https://***.s3.amazonaws.com/e3f0b0a4-8574-11ee-828b-a00749088a1b.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYV4E3J2SZNZVY4XD%2F20231117%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20231117T181259Z&X-Amz-Expires=172800&X-Amz-Signature=0470504b0cd60c07b022443b6ad05b1bb2b2c722aca34ab4ac7ee5327e656a82&X-Amz-SignedHeaders=host&actor_id=62&key_id=0&repo_id=0&response-content-disposition=filename%3De3f0b0a4-8574-11ee-828b-a00749088a1b.tar.gz&response-content-type=application%2Fx-gzip
[2023-11-17 13:13:01] [INFO] Waiting for archive with id 4 generation to finish. Current status: exported
[2023-11-17 13:13:01] [INFO] Archive (metadata) download url: https://***.s3.amazonaws.com/e41d1afe-8574-11ee-89d8-cbbfcfe82d22.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYV4E3J2SZNZVY4XD%2F20231117%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20231117T181300Z&X-Amz-Expires=172800&X-Amz-Signature=2ab0ced5ef9769dc0b4ef7fd1d7cd61a34c0b5a3ba6aacb709c28b5b10962f80&X-Amz-SignedHeaders=host&actor_id=62&key_id=0&repo_id=0&response-content-disposition=filename%3De41d1afe-8574-11ee-89d8-cbbfcfe82d22.tar.gz&response-content-type=application%2Fx-gzip
[2023-11-17 13:13:04] [INFO] Migration in progress (ID: RM_kgDaACQ5YTY2YzZjZS05YjAwLTQ3YWEtYmRjYS02ZTdkZDNmYzk3NTM). State: PENDING_VALIDATION. Waiting 10 seconds...
[2023-11-17 13:13:14] [ERROR] Migration Failed. Migration ID: RM_kgDaACQ5YTY2YzZjZS05YjAwLTQ3YWEtYmRjYS02ZTdkZDNmYzk3NTM
[2023-11-17 13:13:14] [INFO] Migration log available at or by running `gh gei download-logs --github-org *** --github-repo ***`
[2023-11-17 13:13:14] [ERROR] Git archive URL returned a HTTP 403 response. Please make sure that the URL is valid and that the file is accessible
If i try to curl the url it redirects me to an error:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAYV4E3J2SZNZVY4XD</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256
20231117T180019Z
20231117/us-west-2/s3/aws4_request
It seems the generates pre-signed url is not correct. I can confirm that the iam credential I created does have getObject permissions, and can access this file. It's not a permissions issue it's a signing issue. I've tried setting and not-setting:
AWS_REGION="us-west-2"
But it doesn't seem to make any difference in the generated url.
I just tested generating a pre-signed url using the aws cli with the same credential and the generated url does work in that case.
Now on retry it seems to work but the's printed in the logs still return 403 when I try to access? Not sure but closing...
Hi @brokenjacobs . Glad that it's working for you now. Please be sure to rotate that AWS Access Key and make sure that no blob is exposed at the URLs you posted here. This is a public repository so anybody can see this Issue.
Access key has been rotated and I censored the bucket name. Thanks for the heads up!