S3 Function Header tests are failing on ceph.

Question

S3 Function Header tests are failing on ceph.

sumedhak27 opened this issue 3 years ago · comments

Sumedh Anantrao Kulkarni commented 3 years ago

Faulty Tests:

test_object_create_bad_date_unreadable_aws4
test_bucket_create_bad_date_unreadable_aws4
test_object_create_bad_amz_date_unreadable_aws4
test_bucket_create_bad_amz_date_unreadable_aws4

Issue Description
The above mentioned tests are failing on ceph cluster deployed in virtual machine environment.
When the same tests are performed using REST API, we get the appropriate response. (403 Forbidden)

Spunk test FAILURE output

~/s3-tests on  master! ⌚ 5:06:41
$ S3TEST_CONF=./splunk.config virtualenv/bin/nosetests s3tests.functional.test_headers:test_bucket_create_bad_date_unreadable_aws4 -v 2>&1 | tee -a cephtest
.log
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
s3tests.functional.test_headers.test_bucket_create_bad_date_unreadable_aws4 ... FAIL

======================================================================
FAIL: s3tests.functional.test_headers.test_bucket_create_bad_date_unreadable_aws4
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/s3-tests/virtualenv/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/s3-tests/s3tests/functional/test_headers.py", line 1680, in test_bucket_create_bad_date_unreadable_aws4
    eq(e.status, 403)
AssertionError: 400 != 403
>>  raise AssertionError(None or "%r != %r" % (400, 403))

-------------------- >> begin captured logging << --------------------
boto: DEBUG: Using access key provided by client.
boto: DEBUG: Using secret key provided by client.
boto: DEBUG: Using access key provided by client.
boto: DEBUG: Using secret key provided by client.
boto: DEBUG: path=/
boto: DEBUG: auth_path=/
boto: DEBUG: Method: GET
boto: DEBUG: Path: /
boto: DEBUG: Data:
boto: DEBUG: Headers: {}
boto: DEBUG: Host: s3.seagate.com:8000
boto: DEBUG: Port: 8000
boto: DEBUG: Params: {}
boto: DEBUG: establishing HTTP connection: kwargs={'port': 8000, 'timeout': 70}
boto: DEBUG: Token: None
boto: DEBUG: CanonicalRequest:
GET
/

host:s3.seagate.com:8000:8000
user-agent:Boto/2.49.0 Python/2.7.18 Linux/4.18.0-305.3.1.el8_4.x86_64
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20220405T111057Z

host;user-agent;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
boto: DEBUG: StringToSign:
AWS4-HMAC-SHA256
20220405T111057Z
20220405/us-east-1/s3/aws4_request
5c07c5e2e45cd3d2127e4d38a2dc3a6191716c005a7982d67d13a85c3781fad4
boto: DEBUG: Signature:
8d5689ca3e3753b7a376ec8d3b18e9c58ad3bcf4b2c4c180f712ced21f31a802
boto: DEBUG: Final headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Content-Length': '0', 'User-Agent': 'Boto/2.49.0 Python/2.7.18 Linux/4.18.0-305.3.1.el8_4.x86_64', 'Host': 's3.seagate.com:8000:8000', 'X-Amz-Date': '20220405T111057Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=NB71GXG3O7MMBELODQOO/20220405/us-east-1/s3/aws4_request,SignedHeaders=host;user-agent;x-amz-content-sha256;x-amz-date,Signature=8d5689ca3e3753b7a376ec8d3b18e9c58ad3bcf4b2c4c180f712ced21f31a802'}
boto: DEBUG: Response headers: [('transfer-encoding', 'chunked'), ('date', 'Tue, 05 Apr 2022 11:10:57 GMT'), ('connection', 'Keep-Alive'), ('content-type', 'application/xml')]
boto: DEBUG: path=/
boto: DEBUG: auth_path=/
boto: DEBUG: Method: GET
boto: DEBUG: Path: /
boto: DEBUG: Data:
boto: DEBUG: Headers: {}
boto: DEBUG: Host: s3.seagate.com:8000
boto: DEBUG: Port: 8000
boto: DEBUG: Params: {}
boto: DEBUG: establishing HTTP connection: kwargs={'port': 8000, 'timeout': 70}
boto: DEBUG: Token: None
boto: DEBUG: CanonicalRequest:
GET
/

host:s3.seagate.com:8000:8000
user-agent:Boto/2.49.0 Python/2.7.18 Linux/4.18.0-305.3.1.el8_4.x86_64
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20220405T111057Z

host;user-agent;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
boto: DEBUG: StringToSign:
AWS4-HMAC-SHA256
20220405T111057Z
20220405/us-east-1/s3/aws4_request
5c07c5e2e45cd3d2127e4d38a2dc3a6191716c005a7982d67d13a85c3781fad4
boto: DEBUG: Signature:
8d5689ca3e3753b7a376ec8d3b18e9c58ad3bcf4b2c4c180f712ced21f31a802
boto: DEBUG: Final headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Content-Length': '0', 'User-Agent': 'Boto/2.49.0 Python/2.7.18 Linux/4.18.0-305.3.1.el8_4.x86_64', 'Host': 's3.seagate.com:8000:8000', 'X-Amz-Date': '20220405T111057Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=NB71GXG3O7MMBELODQOO/20220405/us-east-1/s3/aws4_request,SignedHeaders=host;user-agent;x-amz-content-sha256;x-amz-date,Signature=8d5689ca3e3753b7a376ec8d3b18e9c58ad3bcf4b2c4c180f712ced21f31a802'}
boto: DEBUG: Response headers: [('transfer-encoding', 'chunked'), ('date', 'Tue, 05 Apr 2022 11:10:57 GMT'), ('connection', 'Keep-Alive'), ('content-type', 'application/xml')]
boto: DEBUG: path=/sumedh-qs8z97io50ska5xwotrltu-1/
boto: DEBUG: auth_path=/sumedh-qs8z97io50ska5xwotrltu-1/
boto: DEBUG: Method: PUT
boto: DEBUG: Path: /sumedh-qs8z97io50ska5xwotrltu-1/
boto: DEBUG: Data:
boto: DEBUG: Headers: {}
boto: DEBUG: Host: s3.seagate.com:8000
boto: DEBUG: Port: 8000
boto: DEBUG: Params: {}
boto: DEBUG: Token: None
boto: DEBUG: CanonicalRequest:
PUT
/sumedh-qs8z97io50ska5xwotrltu-1/

date:
host:s3.seagate.com:8000:8000
user-agent:Boto/2.49.0 Python/2.7.18 Linux/4.18.0-305.3.1.el8_4.x86_64
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20220405T111057Z

date;host;user-agent;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
boto: DEBUG: StringToSign:
AWS4-HMAC-SHA256
20220405T111057Z
20220405/us-east-1/s3/aws4_request
7324daec42f671243e1da37fabf2afb17f4b048ddfee1b7933c8323e5deffe8b
boto: DEBUG: Signature:
4d3b2f052b2260735cbb26938d641a828adbc2c307dc5689516d0b64ce6dc398
boto: DEBUG: Final headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Content-Length': '0', 'User-Agent': 'Boto/2.49.0 Python/2.7.18 Linux/4.18.0-305.3.1.el8_4.x86_64', 'Host': 's3.seagate.com:8000:8000', 'X-Amz-Date': '20220405T111057Z', 'Date': '\x07', 'Authorization': 'AWS4-HMAC-SHA256 Credential=NB71GXG3O7MMBELODQOO/20220405/us-east-1/s3/aws4_request,SignedHeaders=date;host;user-agent;x-amz-content-sha256;x-amz-date,Signature=4d3b2f052b2260735cbb26938d641a828adbc2c307dc5689516d0b64ce6dc398'}
boto: DEBUG: Response headers: [('content-length', '0')]
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
XML: /root/s3-tests/nosetests.xml
----------------------------------------------------------------------
Ran 1 test in 0.360s

FAILED (failures=1)
(virtualenv)

Test output using REST API

The date was set to "\x07" in the request header and appropriate response of 403 Forbidden was recieved.

~/ceph-client ⌚ 6:28:06
$  cd /root/ceph-client ; /usr/bin/env /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.x86_64/bin/java -age
ntlib:jdwp=transport=dt_socket,server=n,suspend=y,address=localhost:43147 -cp /tmp/cp_bbidnu5w7per3qn9hi9pyfa63.j
ar com.seagate.ceph_client.CephAdminAPIClient 
Date: \x07
HeaderString: PUT


\x07
/hehe
Auth: AWS 0555b35654ad1656d804:cotACXLHRe1GRn/DqvlZqV+iLMk=
HTTP/1.1 403 Forbidden

Response Content is: {"Code":"AccessDenied","RequestId":"tx000003dcddc51354026c0-00624ae541-1050-default","HostId":"1050-default-default"}