403 error with version 1.6.3 of boto3

Question

403 error with version 1.6.3 of boto3

adamyala opened this issue 6 years ago · comments

django version: 2.0.2
django-s3-storage version: 0.12.4
boto3 version: 1.6.3

Uploading a file then accessing the url results in a 403 error with the message:

The request signature we calculated does not match the signature you provided. Check your key and signing method.

Last version of boto3 I used that made django-s3-storage work was 1.5.2. This time version 1.6.3 autoinstalled as a dep and it threw the above error.

As always, thank you for such a wonderful module.

Dave Hall · Answer 1 · Tue Mar 06 2018 18:31:04 GMT+0800 (China Standard Time)

I just updated the boto3 dependency to 1.6.3, and I can't reproduce this error. I bet it's something like boto3 1.6.3 uses a different default signature method that your particular endpoint does not support? What S3 region are you using? Meanwhile, you can pin the boto3 version in your requirements.txt file or similar to one that works.

…

On 6 March 2018 at 02:50, Adam Yala ***@***.***> wrote: django version: 2.0.2 django-s3-storage version: 0.12.4 boto3 version: 1.6.3 Uploading a file then accessing the url results in a 403 error with the message: The request signature we calculated does not match the signature you provided. Check your key and signing method. Last version of boto3 I used that made this work was 1.5.2. This time version 1.6.3 autoinstalled as a dep and it threw the above error. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#74>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJFCIU_4BG7f3rirDujrmTGBx4djtNGks5tbflwgaJpZM4SeBTj> .

Adam Yala · Answer 2 · Wed Mar 14 2018 07:21:49 GMT+0800 (China Standard Time)

I sat down to recreate today and the problem went away. No idea why. Apologies for eating up some of your time.

If the issue comes back I'll reopen.

Dave Hall · Answer 3 · Thu Mar 15 2018 04:40:30 GMT+0800 (China Standard Time)

No worries!

…

On 13 March 2018 at 23:21, Adam Yala ***@***.***> wrote: I sat down to recreate today and the problem went away. No idea why. Apologies for eating up some of your time. If the issue comes back I'll reopen. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#74 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJFCAylEmKQR6kcFzHUscIbN03jghK_ks5teFSOgaJpZM4SeBTj> .

Adam Yala · Answer 4 · Sat Oct 20 2018 05:38:18 GMT+0800 (China Standard Time)

@etianen I'd like to reopen this issue and help debug the underlying cause. I have the latest boto3 version installed (1.9.27) and this bug reappeared. I downgraded to 1.5.2 per my note above to see if that would fix it, and it did fix it.

What information can I supply to start our research?

Adam Yala · Answer 5 · Sat Oct 20 2018 05:42:26 GMT+0800 (China Standard Time)

Below are my settings:

DEFAULT_FILE_STORAGE = 'django_s3_storage.storage.S3Storage'
STATICFILES_STORAGE = 'django_s3_storage.storage.StaticS3Storage'
# The AWS region to connect to.
AWS_REGION = 'us-west-2'
# The AWS access key to use.
AWS_ACCESS_KEY_ID = 'XXXX'
# The AWS secret access key to use.
AWS_SECRET_ACCESS_KEY = 'XXXX'
# The name of the bucket to store files in.
AWS_S3_BUCKET_NAME = AWS_S3_BUCKET_NAME_STATIC = 'XXXXX'

Note that this only seems to be failing locally (while developing on my laptop). When running on a server things seem fine.

Dave Hall · Answer 6 · Tue Oct 23 2018 00:42:47 GMT+0800 (China Standard Time)

I'm afraid that I'm not really able to look into this myself at the moment. If you can submit a pull request that fixes the issue for you, I'd be happy to review and merge.

…

On Fri, 19 Oct 2018 at 22:42, Adam Yala ***@***.***> wrote: Below are my settings: DEFAULT_FILE_STORAGE = 'django_s3_storage.storage.S3Storage' STATICFILES_STORAGE = 'django_s3_storage.storage.StaticS3Storage' # The AWS region to connect to. AWS_REGION = 'us-west-2' # The AWS access key to use. AWS_ACCESS_KEY_ID = 'XXXX' # The AWS secret access key to use. AWS_SECRET_ACCESS_KEY = 'XXXX' # The name of the bucket to store files in. AWS_S3_BUCKET_NAME = AWS_S3_BUCKET_NAME_STATIC = 'XXXXX' Note that this only seems to be failing locally (while developing on my laptop). When running on a server things seem fine. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#74 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJFCLl0dxzIA7jSM1aUBh3xAg4VSYL_ks5umkdCgaJpZM4SeBTj> .

Tjorriemorrie · Answer 7 · Sat Feb 16 2019 14:51:42 GMT+0800 (China Standard Time)

I'm also getting a 403. Do I need to change the settings of the bucket or anything else?

vesatoivonen · Answer 8 · Thu Apr 04 2019 16:13:40 GMT+0800 (China Standard Time)

The same thing was happening to me yesterday, but once i started to debug the issue today it seems to have vanished.
I did not change any packages in between.

boto3==1.9.127
Django==2.2
django-s3-storage==0.12.4

Roderick Schaefer · Answer 9 · Mon Feb 17 2020 18:41:55 GMT+0800 (China Standard Time)

Same here. Pinning boto3 to 1.5.2 solved it immediately.

Dave Hall · Answer 10 · Tue Feb 18 2020 17:11:03 GMT+0800 (China Standard Time)

Ideally someone would reproduce this issue using the latest boto and make a PR. Then we can specify a minimum version of boto3 in `setup.py`.# Until then, pinning boto3 to a working version is the correct workaround.

…

On Mon, 17 Feb 2020 at 10:42, Roderick Schaefer ***@***.***> wrote: Same here. Pinning boto3 to 1.5.2 solved it immediately. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#74?email_source=notifications&email_token=AABEKCGRNQIUHVBEUD6G22TRDJSXHA5CNFSM4ETYCTR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL55IIY#issuecomment-586929187>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKCEMHVC3MF3HU6NTXATRDJSXHANCNFSM4ETYCTRQ> .

Wes Kendall · Answer 11 · Thu Aug 20 2020 02:38:36 GMT+0800 (China Standard Time)

@etianen and others, I had this error until I found this boto/boto3#1644 (comment)

To get rid of this error, set the following settings:

AWS_S3_ADDRESSING_STYLE = 'path'

Now I successfully can view objects with boto3=1.14.45 and django-s3-storage="0.13.3"

@etianen do you think this should be changed to the default setting? I'm happy to open a PR. A better solution might be adjusting this based on the boto3 version, but I'm not sure if I will have time to make that PR

Dave Hall · Answer 12 · Mon Aug 24 2020 17:19:11 GMT+0800 (China Standard Time)

Do you have any idea why the current `auto` setting fails, but `path` does not? That sounds very much like a problem with boto3.

…

On Wed, 19 Aug 2020 at 19:38, Wes Kendall ***@***.***> wrote: @etianen <https://github.com/etianen> and others, I had this error until I found this boto/boto3#1644 (comment) <boto/boto3#1644 (comment)> To get rid of this error, set the following settings: AWS_S3_ADDRESSING_STYLE = 'path' Now I successfully can view objects with boto3=1.14.45 and django-s3-storage="0.13.3" @etianen <https://github.com/etianen> do you think this should be changed to the default setting? I'm happy to open a PR. A better solution might be adjusting this based on the boto3 version, but I'm not sure if I will have time to make that PR — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#74 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKCC2SRBENL6U3FTBFUTSBQL3VANCNFSM4ETYCTRQ> .

Tomasz Wójcik · Answer 13 · Thu Jul 22 2021 17:48:32 GMT+0800 (China Standard Time)

I struggled with lots of errors but finally managed to make it work (for generate_presigned_post).

I get my client from (if you need it)

client = S3Boto3Storage().bucket.meta.client

And those are the settings you need to experiment with

AWS_S3_SIGNATURE_VERSION = "s3v4"  # required for some regions
AWS_S3_ADDRESSING_STYLE = 'path'  # needed only in some cases, depending on your boto3 version
AWS_S3_REGION_NAME = ...  # might be necessary if your region is non-default

boto3==1.18.4
django-storages==1.11.1