Continuously call of TransferManager methods makes not to work blob accesses
tsbttk opened this issue · comments
Which service(blob, file, queue, table) does this issue concern?
blob
Which version of the SDK was used?
10.1.0, 10.5.0
What problem was encountered?
In our application, we use azure-storage-blob-10.1.0.jar to access to the Azure blob service.
We use "TransferManager.downloadBlobToFile(downloadFileChannel, blob, null, null).blockingGet()" method
and "TransferManager.uploadFileToBlockBlob(fileChannel, blob, 8 * 1024 * 1024, null).blockingGet()" method.
After we call these methods continuously (for about 13 days or call these methods thousands times), we can not access to the blob service via these methods. After certain moment these methods become to return timeout.
We could get a heapdump while the phenomenon was happening. From the heapdump we notice SharedChannelPool#requests instance that is instance of ConcurrentLinkedDeque has many requests that is not transacted.
we also notice it is because size of SharedChannelPool#leased instance is 64, and SharedChannelPool#poolSize is also 64.
So SharedChannelPool thread stops at L.109
while (leased.size() >= poolSize && !closed) {
sync.wait(); // L.109
}
(We know now 64 is the number of core (2) * 32 from source code of libraries.)
As a trial, while the application is running, we stop calling thse download and upload methods.
After we stop calling these download and upload methods, the size of SharedChannelPool#leased map stays 64.
So we think that something like leak may exist in SharedChannelPool#leased map.
We want to know how we can call these 2 methods continuously.
Have you found a mitigation/solution?
No, not yet.