CODAIT / stocator

Stocator is high performing connector to object storage for Apache Spark, achieving performance by leveraging object storage semantics.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

IAM token is not getting updated on 1.0.36-ibm-sdk

benzim opened this issue · comments

I am seeing the same behavior from #152 on branch 1.0.36-ibm-sdk. Once the first token used expires, trying to access the bucket, even after setting fs.cos.<service>.iam.token to a new valid token, results in this error.

Caused by: java.io.IOException: com.ibm.stocator.thirdparty.cos.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 25fa4ea4-74d1-4378-a822-89cf38bff151), S3 Extended Request ID: null
	at com.ibm.stocator.fs.cos.COSAPIClient.getFileStatus(COSAPIClient.java:602)
	at com.ibm.stocator.fs.ObjectStoreFileSystem.getFileStatus(ObjectStoreFileSystem.java:515)

@benzim you are not suppose to update IAM tokens manually. This is done with COS SDK internally

According to the readme,

An optional, it is possible to provide existing token instead of using API key. Instead of providing fs.cos.myCos.iam.api.key, Stocator supports fs.cos.myCos.iam.token that may contain value of the existing token. When token is expired, Stocator will throw 403 exception. It's the user responsibility to provide long activation token or re-create token outside of Stocator.

My code is getting a token from a user and then setting fs.cos.myCos.iam.token.

@benzim yeah...but this is very tricky. It's documented indeed, but it's not strait forward to use it. It's better advised to use api key and not provide tokens outside