Max memory allocation error

[Quinton Ashley]

I get this error log only when using large images (1.7MB image 2250 × 3000 in this case) with the detailed preset. Is it trying to allocate a svg file that would be 1.4GB?

<--- Last few GCs --->


[17629:0x102803200]    29323 ms: Mark-sweep 1418.4 (1575.9) -> 1418.4 (1575.9) MB, 218.4 / 0.0 ms  allocation failure scavenge might not succeed
[17629:0x102803200]    29547 ms: Mark-sweep 1418.4 (1575.9) -> 1418.3 (1559.9) MB, 222.8 / 0.0 ms  last resort GC in old space requested
[17629:0x102803200]    29761 ms: Mark-sweep 1418.3 (1559.9) -> 1418.3 (1559.9) MB, 213.8 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x333705425ee1 <JSObject>
    2: /* anonymous */ [/Users/qashto/Documents/web/video-art/node_modules/imagetracerjs/imagetracer_v1.2.3.js:~352] [pc=0x12f1b38a67e7](this=0x33378a48cee9 <ImageTracer map = 0x33372c76cbb1>,ii=0x333721e85f69 <Object map = 0x3337fee9d9a9>)
    4: /* anonymous */ [/Users/qashto/Documents/web/video-art/node_modules/imagetracerjs/imagetracer_v1.2.3.js:101] [bytecode=0x3337e887b179 offset=46](this=...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/local/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/usr/local/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
 4: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/bin/node]
 5: v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
 6: 0x12f1b370463d
 7: 0x12f1b37c1973
 8: 0x12f1b38a67e7
Abort trap: 6

[András Jankovics]

Hi,

Yes, I know this problem, it's mentioned in Process overview :
"Error handling: there's very little error handling now, Out of memory can happen easily with big images or many layers."

Calculation:
colorquantization creates a 1px frame on each side of the image, so the indexed array size is 2252 * 3002 .
layering creates as many layers as numberofcolors, which is 64 with the Detailed preset.
So 2252 * 3002 * (1 + 64) * 8 {64bit JavaScript Numbers} = 3 515 462 080 bytes , which is over 3GB before we even begin with the tracing.

I recommend changing the memory limits of the V8 engine:

node --max-old-space-size=8192 yourcode.js

(it's in megabytes)
(There might have been trouble with this earlier, but the latest v9.7.1 Documentation lists this as valid. https://nodejs.org/api/cli.html#cli_node_options_options)

In theory, a new layering function could be created which outputs only 1 layer each time. This layer can be processed with pathscan, internodes and tracepath and discarded afterwards. This would improve memory consumption, but probably impact performance as layering would be called numberofcolors times. I'll think about implementing this.

There's no built in checks for image size or memory usage because:

• I haven't found a good solution for these
• It might impact performance
• Keeping it simple: I try to avoid complexity when it works most of the time

Please give me a feedback if this helps or not. You can attach the input image if it's public, so I can test it. ☺

[Quinton Ashley]

Thank you for the detailed response! I'm glad that I can set node's memory output higher, I didn't know it was given so little by default! I also think it's good to avoid the complexity of trying to error check for memory usage, you're right it's unnecessary. I am very impressed with the results I'm getting with your script when using smaller images. I was merely trying to test the limits with a large picture. When I come across an incredible project like yours I always think GitHub should have some sort of compliments page in addition to the issues and stars. A star is so impersonal!

[András Jankovics]

I created a sequential layering method in version 1.2.4 , and hope it helps reducing the memory usage.

[Stav Geffen]

Am facing this issue with imagetracerjs@1.2.5

--max-old-space-size=8192 is set

Using these settings:

{ blurradius: 4, pathomit:8, roundcoords:2, ltres:0.5, qtres:0.5, numberofcolors:32 }

Getting the same error for some 500x450 images

[András Jankovics]

Hi,

Setting both ltres and qtres to small values (<1) might force too much recursion in the fitseq divide-and-conquer algorithm (in 5. Tracing). I recommend either ltres:0.5 ; qtres:1 (for more round shapes) or ltres:1 ; qtres:0.5 (for more straight shapes).

I hope this helps. You can also reopen the Issue or attach example images.

[Stav Geffen]

Unfortunately both of these recommendations do not change the outcome.

[András Jankovics]

Please reopen this Issue or open a new one and attach your code and image that has problem.