Prisma.sql and queryRaw seem to be leaking memory

[Paulo Cesar]

Bug description

We recently updated some queryRawUnsafe to use queryRaw and escaping nested queries with Prisma.sql and now we have a constant climbing in memory and restart every hour or so.
The queries are very complex, but they weren't causing any major leaks before this change.

Things that could be causing it (but can't easily reproduced):

• Inserting nested Prisma.sql using an array (eg array.push(Prisma.sql))
• Using a class to build a query instead of using a simple function without any free/dispose methods (but was working fine as-is before)
• "Global" Prisma.sql constants that get instantiated outside of functions once, but referenced multiple times inside functions and classes

How to reproduce

Expected behavior

No response

Prisma information

// Add your schema.prisma

// Add your code using Prisma Client

Environment & setup

• OS: Debian Bookworm
• Database: PostgreSQL
• Node.js version: 20.8.0

Prisma Version

prisma                  : 5.4.2
@prisma/client          : 5.4.2
Current platform        : debian-openssl-3.0.x
Query Engine (Node-API) : libquery-engine ac9d7041ed77bcc8a8dbd2ab6616b39013829574 (at node_modules/@prisma/engines/libquery_engine-debian-openssl-3.0.x.so.node)
Schema Engine           : schema-engine-cli ac9d7041ed77bcc8a8dbd2ab6616b39013829574 (at node_modules/@prisma/engines/schema-engine-debian-openssl-3.0.x)
Schema Wasm             : @prisma/prisma-schema-wasm 5.4.1-2.ac9d7041ed77bcc8a8dbd2ab6616b39013829574
Default Engines Hash    : ac9d7041ed77bcc8a8dbd2ab6616b39013829574
Studio                  : 0.494.0
Preview Features        : fullTextSearch

[Jan Piotrowski]

What change exactly did you do in your app? I am not sure I fully understood that.

Just to mention it, although I realize you probably tried and couldn't: We will need some kind of reproduction to be able to dig into this. Without it, we are trying to fix something that we do not see and can not confirm if it is indeed fixed. That will be challenging.

[Paulo Cesar]

@janpio it was caused exclusively by nested Prisma.sql string template substitutions. since it uses the leaky sql-template-tag library and it seems that queryRaw also keeps the reference of those instances while performing a query. the strings balloon up and never gets GCed.
We solved the issue removing all calls to Prisma.sql, but you can close this if you want

[Jan Piotrowski]

No, this is great - optimally you could even provide an example that we can reproduce and then use to minimize or fix this? Thanks!

[Paulo Cesar]

@janpio this is something that I used just to check the heap snapshot to confirm my suspicion:

import { Prisma, PrismaClient } from '@prisma/client';
import { setTimeout } from 'node:timers/promises';

const prisma = new PrismaClient();
const PRISMA_EMPTY_STRING = Prisma.sql` `;

const f = async (elements) => {
  const els = elements.map((element) => Prisma.sql`${element} as "${element}"`);
  els.push(PRISMA_EMPTY_STRING);
  return await prisma.$queryRaw`SELECT ${els}`;
};

for (let i = 0; i < 200; i++) {
  const args = Array.from({ length: 20 }, () => 0).map((_, index) =>
    index % 2 === 0 ? (Math.round(Math.random() * 100) + 36).toString(36) : index,
  );

  console.log('going to query');
  console.dir(
    await f(args)
  );
  console.log('queried');
  await setTimeout(10);
}

debugger;
await setTimeout(2**30); // keep process alive

[Joël Galeran]

@pocesar I just tried with our dev version and could not reproduce. See reproduction attempt in #21734

Could you provide more information about this memory leak you identified, how should I reproduce and observe the leak?
Sharing any data on this would help, for example, what size of memory leak are we talking about, screenshots are welcome if that helps.

Additionally, try using the latest version 5.5.2 or the dev tag to see if that is reproducible.

[Joël Galeran]

Note that I tried with different versions of Node.js in GitHub Actions in #21734

Local run:

No memory leak detected
Memory growth rate: 0.14974214974214975 bytes / iteration, which is below threshold

CI run: Node.js 16.20.2

Running test 21512-queryRaw
No memory leak detected
Memory growth rate: 0.7731143091143091 bytes / iteration, which is below threshold

Node.js 18.18.2

Running test 21512-queryRaw
No memory leak detected
Memory growth rate: 0.17308529308529308 bytes / iteration, which is below threshold

Node.js 20.9.0

Running test 21512-queryRaw
No memory leak detected
Memory growth rate: 0.11187543987543988 bytes / iteration, which is below threshold

The threshold is defined as:

const GROWTH_RATE_THRESHOLD_IN_BYTES = 10

I can see that the "Memory growth rate" from these runs is lower in Node.js v18, which is also lower in Node.js v20.
@pocesar Are these results similar to what you're seeing? I'm curious to know more about what you can observe.

[Joël Galeran]

@pocesar I attempted another reproduction using the Prisma version you specified in this issue 5.4.2 and could still not reproduce.

Here is the setup I used:
https://github.com/Jolg42/repro-21512

• Prisma 5.4.2
• Node.js v20.8.0
• Important to note, I used NODE_OPTIONS="--max-old-space-size=200" to put a limit for V8, it helps to show that rss is reclaimed here, even if there is still free memory available on the machine.

Could you provide more information?
Can you reproduce it in a minimal reproduction? Feel free to fork my repository.

[Jan Piotrowski]

@pocesar Did you see Joel's comment? We need your help to reproduce this. Thanks.

[Paulo Cesar]

@janpio @Jolg42 sorry, been kinda crazy lately. we created a separate deployment that uses prisma heavily, but we switched from using Prisma.sql to use sql-template-strings completely and we solved the issue but I can't tell how to reproduce it currently

[Joël Galeran]

@pocesar Happy new year 😉

I am closing this, as we're missing a reproduction.

If you have any information that would help, let us know, we'll be happy to re-open this issue and attempt a reproduction based on your input.