UTIF.decodeImage taking too much memory
varun-raghavendra opened this issue · comments
I'm trying to decode 5184x3456 resolution TIFF images. This makes my laptop memory to spike up to 12gb from 3gb. More details about the files:
- Number of TIFF files - 20
- Size of each TIFF file ~ 100MB
- Resolution of each TIFF file - 5184x3456
- Black and white
Your TIFF file (106 MB) contains 50 identical images. Are you aware of this? If you need just one image, call decodeImage just on one IFD.
Eachi pixel takes 4 Bytes after the decompression.
A 5184x3456 image takes 71 MB of RAM. But your file has 50 such images (3.5 GB of RAM). If you keep several such files in memory.
I am aware that my TIFF file contains 50 identical images. This is a test document I used. I have a use case where I need to split such documents.
I see that there's another method in UTIF called encodeImage. Does this encode to a TIFF? Is there any way to compress this TIFF file?
A 5184x3456 image takes 71 MB of RAM. But your file has 50 such images (3.5 GB of RAM). If you keep several such files in memory.
Any way to reduce this 71MB size?
Avoid using decodeImage(), make your own decoding function instead.
Your image is 1 bit per pixel, but decodeImage makes it 4 bytes per pixel (32x larger).
Oh okay, so in this case, I should be using the IFD array I obtain as output from UTIF.deocde right?
It will be helpful if you could point me to some source which will help me understand this decoding process better so that I can implement it myself.
Also, UTIF.decode already produces an IFD array. So why can't this itself used to create a new PNG?
@varun-raghavendra Look at some of the demo code I wrote a few years ago when I was looking at using UTIF.js in an application. You'll find this at https://github.com/johnthad/utif-demo
Note that UTIF.js has been updated a few times since this was written. I recommend you use the latest version of UTIF.js on GitHub.
Actually, decodeImage() should still use 1 bit per pixel, so it should be safe to call it. But if you call UTIF.toRGBA8(), this turns the image into 4 Bytes per pixel (32x more).
A little bit of computation: Your image is 5184x3456 pixels. With 1 bit per pixel, it is 2,239,488 B per image. 111,974,400 per 50 images.
Your TIFF file is 111,984,308 Bytes. So your file is the uncompressed images + 9,908 Bytes of additional data.
Do you know how to analyze the usage of RAM of each browser tab? Please, show us some proof that UTIF.js uses more RAM than it should.
@photopea I'm using the activity monitor on my Macbook to find the RAM usage.
I think you're right. It looks like it's mainly the decompressed images (PNGs) that are taking up more RAM, and not any of the UTIF algorithms themselves.
One more thing, what is the time complexity of the algorithm used by UTIF for decode and decodeImage? Can you confirm that UTIF.decode and UTIF.decodeImage functions themselves don't have any impact on the memory other than the final output files?
@johnthad Thanks, I'll take a look.
are you asking about time complexity or the memory complexity?
Your TIFF file has no compression, so it is "decompressed" in linear time. The decompressed data has the same size as compressed (111 MB). If you call toRGBA8(), they become 32x larger.
I'm asking about memory. What would be the memory complexity if the compression was different? Which compression gives the highest memory complexity? Also now that you mentioned it, same question about time complexity.
The TIFF format supports many compression methods. But if you export a TIFF file with UTIF.js, there is no compression applied.