i3s converter extract to cloud blob storage will hang
tom62291 opened this issue · comments
Hi, sometimes the i3s converter extract process will hang when the destination is cloud storage.
It just hanged like below and the progress will not go anymore.
Thanks for bringing this up @tom62291
Can you provide a sample of the data you are trying to extract to cloud storage?
Also, which cloud service are you trying to extract to?
It is a large integrated mesh SLPK around 110 GB and I am using Azure blob storage.
Sorry that I can not provide sample data due to NDA.
@tom62291
While testing, we have seen similar behavior when credentials are incorrect and when the container is incorrect. Can you please verify that you have write permissions to the container, that the container exists, and the correct credentials are being used.
The credential should be correct as I can see that 1134124 files in the blob container
@tom62291
Does your extraction hang indefinitely (and you cancel) or do you eventually receive an output?
It hang indefinitely
any news?
if no, can provide any guidelines to unpack the SLPK to i3srest which stored in the local drive first, and then use azcopy command provided by Microsoft to upload the local i3srest folder to azure blob storage?
@tom62291
We believe this could be a network issue. How many threads are being used to process the SLPK to Azure? If you are using multiple threads, can you try using a single thread?
Does the converter get stuck at the same amount completed when using a single thread?
There is no easy way to perform unpacking locally. You could extract to ESLPK and then would have to rename all the file names to conform to I3SRest format.
@tom62291
We have made updates to the i3s_converter which hopefully addressed these issues.