API Does'nt work thru an HTTPS Proxy
fberzolla-ibm opened this issue · comments
The API does not work if we are using an HTTPS Proxy !
To be precise The IAM authentication part of the API does not work if we are using an HTTPS Proxy.
The method com.ibm.cloud.objectstorage.oauth.DelegateTokenProvider.retrieveToken() uses a direct socket connection and therefore doesn't go thru the HTTPS proxy !
We have a customer that cannot use a SOCKS proxy for security reasons !
Is there a way to "customize" the API in order to go thru the HTTPS proxy for the IAM authentication part ?
Hi @fberzolla-ibm I presume you are using the DelegateTokenProvider for Aspera transfers. You can overwrite the default config on the provider like below;
DelegateTokenProvider asperaTokenProvider;
asperaTokenProvider = new DelegateTokenProvider(<API_KEY>).withIamEndpoint(<IAM_URL>);
DefaultTokenManager tokenManager = new DefaultTokenManager(asperaTokenProvider);
tokenManager.setIamEndpoint(<IAM_URL>);
AsperaTransferManager asperaTransferManager = new AsperaTransferManagerBuilder(<API_KEY>, ).withTokenManager(tokenManager).build();
Let me know if this works for you
Hi @fberzolla-ibm I presume you are using the DelegateTokenProvider for Aspera transfers. You can overwrite the default config on the provider like below;
DelegateTokenProvider asperaTokenProvider;
asperaTokenProvider = new DelegateTokenProvider(<API_KEY>).withIamEndpoint(<IAM_URL>);DefaultTokenManager tokenManager = new DefaultTokenManager(asperaTokenProvider);
tokenManager.setIamEndpoint(<IAM_URL>);AsperaTransferManager asperaTransferManager = new AsperaTransferManagerBuilder(<API_KEY>, ).withTokenManager(tokenManager).build();
Let me know if this works for you
Hi @seamy49
No I'm not using DelegateTokenProvider for Aspera transfers!
I'm just using the example coming from the README.
https://github.com/IBM/ibm-cos-sdk-java/blob/master/README.md
I'm not using not setting the TokenManager myself ! It is Embeded in the BasicIBMOAuthCredentials class !
AWSCredentials credentials;
if (endpoint_url.contains("objectstorage.softlayer.net")) {
credentials = new BasicIBMOAuthCredentials(api_key, service_instance_id);
} else {
String access_key = api_key;
String secret_key = service_instance_id;
credentials = new BasicAWSCredentials(access_key, secret_key);
}
ClientConfiguration clientConfig = new ClientConfiguration().withRequestTimeout(5000);
clientConfig.setUseTcpKeepAlive(true);
AmazonS3 cos = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(credentials))
.withEndpointConfiguration(new EndpointConfiguration(endpoint_url, location)).withPathStyleAccessEnabled(Boolean.TRUE)
.withClientConfiguration(clientConfig).build();
If you run the example and if you set up an HTTPS Proxy then it fails ! Because the BasicIBMOAuthCredentials uses behind the scene the DelegateTokenProvider.
@fberzolla-ibm BasicIBMOAuthCredentials uses DefaultTokenProvider, the delegate provider is for Aspera transfers. Are you looking to set a different endpoint to send IAM token requests through your proxy?
@smcgrath-IBM
What I want to achieve is very simple. I want to run the sample CosExample that you provide in the README.
This work fine with my own parameters
SDKGlobalConfiguration.IAM_ENDPOINT = "https://iam.bluemix.net/oidc/token";
String bucketName = "mazars";
String api_key = "MY API KEY";
String service_instance_id = "cf6885c9-5a79-453f-9657-afe64e1c245d";
String endpoint_url = "https://s3.eu-geo.objectstorage.softlayer.net";
String location = "eu-geo";
But If I set an HTTPS proxy at the JVM layer
System.setProperty("https.proxyHost", "myProxyHost");
System.setProperty("https.proxyPort", "8080");
The the sample code does not work anymore !
It seems that the code that obtain the token from the iam URL https://iam.bluemix.net/oidc/token
(located in the class com.ibm.cloud.objectstorage.oauth.DelegateTokenProvider.retrieveToken()) Does not use the JVM proxy !
@fberzolla-ibm can you enable debug logging with log4j config & attach the output?
@fberzolla-ibm I just ran some tests on a proxy I set up locally, mitmproxy for mac, I can send requests to both staging & production bluemix using IAM authentication. It is quite possible some config is needed on your proxy.
@smcgrath-IBM I'm also using mitmproxy for mac.
If I set-up the client to use mitmproxy only for HTTPS requests then I can see on the mitmproxy log only request for https://s3.eu-geo.objectstorage.softlayer.net !
No request for https://iam.bluemix.net/oidc/token are going through the Proxy !!
If I set-up the client to use mitmproxy for HTTPS and also SOCKS proxy then I can see on the mitmproxy logs request for https://s3.eu-geo.objectstorage.softlayer.net and also for https://iam.bluemix.net/oidc/token
That mean that the requests for https://iam.bluemix.net/oidc/token are not using the HTTPS proxy !
Can you confirm that ? Thanks
Hi @fberzolla-ibm on analysing mitmproxy logging, I can see only S3 calls using the proxy as you have found, not IAM tokens. I have created an internal issue to track this CSAFE-47973. As a workaround you can use hmac authentication i.e accessKey & secretKey. You can retrieve them with these steps;
https://console.bluemix.net/docs/services/cloud-object-storage/iam/service-credentials.html?cm_mc_uid=66034702458014896630053&cm_mc_sid_50200000=94967721537441544870#service-credentials?cm_sp=dw-bluemix-_-nospace-_-answers
Hi @fberzolla-ibm , a fix for this issue has been released in the latest version of the Java SDK 2.4.2, please let us know if this resolves your issue.
Hi @fberzolla-ibm did you try the latest SDK, is it ok to close this off?
closing as fix supplied in 2.4.2 release