Faced with memory leaks when work with autoPagination + pollingLive
alohamonius opened this issue · comments
Hello community, I hope you can guide me to the solution.
So I work with dexes to fetch information about ALL pairs/pools. I use pollingLive with auto pagination plugins for graph clients. As a result, my usedHeap increased on 10Mb/s and I got out of memory exception, or every fetch increase the heap on the 100MB. I have commented all my code and only left graph clients, and have the same problem.
So I have few questions about:
- Can I clear data inside mesh?
- For now I using one MeshInstance per dex(v2,v3), with Repeater object going through all results. Maybe I did wrong here?
"@graphprotocol/client-cli": "^2.2.21",
"@graphprotocol/client-polling-live": "1.1.1",
"graphql": "16.6.0",
Could you create an example repo for us to reproduce the leak?
https://github.com/alohamonius/graph-memory-leak
Check readme.
I don't find a way how to put sushi/uni in one client, so I worked with them from folders.
One thing I noticed in my first look;
These are internal functions;
https://github.com/alohamonius/graph-memory-leak/blob/master/index.ts#L20
We don't recommend to use them in production. You should use the instance from .graphclient (artifacts).
I added version2 without parseConfig.
The same behavior,
3:56:05 PM heapUsed │ 353.22
4:03:55 PM heapUsed │ 533.65
@ardatan Did you check v2 without internal method?
So what I can do? Can I somehow clean the data inside?
UDP: Really, looks like it stop to consume a lot of heap, GC works well. So, really looks like the problem was only in the usage of the internal parseConfig method. With 2 clients, heapUsed jumping between 1.1GB - 2.5GB, not the best, but better than before.
@alohamonius Why did you reopen it?
@alohamonius Why did you reopen it?
Hello. Because of something really strange with live polling. App eats memory for one hour up to 4GB. But when I use 'execute' with cron-job results are much better. But no problem, I will close again. Thanks)