KnicKnic / WASM-ImageMagick

Webassembly compilation of https://github.com/ImageMagick/ImageMagick & samples

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Error 24: Too many open files

blipk opened this issue · comments

commented

Hi, I'm getting the following error once I process about 3,000 images:

image

Stack:

Error
    at new ErrnoError (https://localhost:1234/magick.js:130:81097)
    at Object.nextfd (https://localhost:1234/magick.js:130:61055)
    at Object.createStream (https://localhost:1234/magick.js:130:61657)
    at Object.open (https://localhost:1234/magick.js:130:73081)
    at Object.writeFile (https://localhost:1234/magick.js:130:77835)
    at processFiles (https://localhost:1234/magick.js:69:10)
    at onmessage (https://localhost:1234/magick.js:127:3)

This is my code:

let imcontext = im.newExecutionContext(/*imcontext*/)
async function featherMagick(file, pngBuffer, allFrames, frame, padFrame, blurSigma) {
    imcontext = im.newExecutionContext(imcontext)
    Object.values(await imcontext.getAllFiles()).forEach(f => imcontext.removeFiles(f.name))

    const infiles = [{ name: "input.jpg", content: file.content },
    { name: `polygon${padFrame}.png`, content: pngBuffer }]
    let cmds = []

    let scale = 100 / parseInt(allFrames[frame]['Scale Data'].x)
    //scale = scale - Math.floor(scale)

    if (normalizeScale)
        scale = scale.toFixed(2).toString().replace('.', '')
    else scale = 100

    //cmds.push(`convert -size ${imgwh[0]}x${imgwh[1]} xc:transparent -fill white -stroke black -draw 'path "M${path}z"' polygon${padFrame}.png`);
    //cmds.push(`convert input.jpg[${scale}%] -matte mask.png[${scale}%] -compose DstIn -composite -alpha set -virtual-pixel transparent -channel A -blur 0x${blurSigma-2} -level 50%,100% +channel out${padFrame}.png`);
    cmds.push(`convert input.jpg -matte polygon${padFrame}.png -compose DstIn -composite -alpha set -virtual-pixel transparent -channel A -blur 0x${blurSigma - 2} -level 50%,100% -trim +repage -scale ${scale}% +channel out${padFrame}.png`);
    let count = 0;
    let outfiles = []
    for (const cmd of cmds) {
        count++;
        const { outputFiles, results, stdout, stderr, errors, exitCode } = await imcontext.execute({
            inputFiles: infiles,
            commands: [cmd]
        })
        outfiles.push(outputFiles)
        if (stdout.join('\n')) if (debug) console.log('Output:' + stdout.join('\n'))
        if(exitCode !== 0) if (debug) console.error('Errors: ' + stderr.join('\n'))
    }
    
    return outfiles.flat()
}

Previously I tried with a new context on each run and without removing the files and still the same issue:

async function featherMagick(file, pngBuffer, allFrames, frame, padFrame, blurSigma) {
    let imcontext = im.newExecutionContext()
    ....
}

Is there anything the API exposes to clean up the files properly?

Thanks.

edit:
I also tried passing another import to each call of the function but same issue:

const im = await import('wasm-imagemagick')
featherMagick(im, file, pngBuffer, allFrames, frame, padFrame, blurSigma)

...

async function featherMagick(im, file, pngBuffer, allFrames, frame, padFrame, blurSigma)
commented

Solved the issue by wrapping in my own workers that only process X amount of images at a time.