Broken multipart upload in 2.4.1
elder-cb opened this issue · comments
Using TransferManager
to upload a File
with version 2.4.1 throws exception:
com.ibm.cloud.objectstorage.SdkClientException: Failed to mark the file position
at com.ibm.cloud.objectstorage.internal.ResettableInputStream.mark(ResettableInputStream.java:148)
[...]
Caused by: java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:122)
at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:265)
at com.ibm.cloud.objectstorage.internal.ResettableInputStream.mark(ResettableInputStream.java:146)
Sample code:
// test file created with head -c $((15*1024*1024)) /dev/random > fileLargerThanOnePart.rnd
File localFile = new File("fileLargerThanOnePart.rnd");
PutObjectRequest putRequest = new PutObjectRequest(bucketName, key, null, metadata);
putRequest.setFile(localFile);
Long partSize = 5 * 1024 * 1024L;
TransferManager transferManager = TransferManagerBuilder.standard() //
.withS3Client(this.client) //
.withMinimumUploadPartSize(partSize) //
.withMultipartCopyPartSize(partSize) //
.withMultipartUploadThreshold(partSize) //
.build();
Upload upload = transferManager.upload(putRequest);
upload.waitForCompletion();
Expected:
Successful upload, as it happens with version 2.0.0
Observed:
java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:122)
at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:265)
Version tested:
v2.4.1
@elder-cb I'm looking into this now.
Is there a reason you're using 2.4.1? The current version is 2.4.2
@elder-cb do you have any more informaiton on this? I'm currently unable to recreate the issue.
@paul-carron
I've uploaded a sample project to replicate the issue, updating also to version 2.4.2 (I was using 2.4.1 'cause it was the latest when I started coding with it and never updated): https://github.com/elder-cb/cos-sdk-issue-16
Note:
I am working on MacOS with oracle java 8, but had seen the identical issue in a docker image based on official IBM Liberty docker image (so with IBM JDK 8) and our test server (Centos with IBM JDK).
Ticket CSAFE-51255, has been raised to track this issue internally.
FYI, I've added docker support in the sample code using ibmcom/ibmjava:8-sdk
base image
The fix for this issue has been released within the 2.4.4 version of the Java SDK. Please confirm if this release resolves your issue.
Tested again with the sample code after updating the dependency version: it now works fine, thank you! :)