Memory leaks in 4.1.1?

[danielpater]

Hi Joannis,

thank you for your great packages and for adding Codable encoder/decoder support to your BSON library recently.

Unfortunately, I'm running into memory issues with MongoKitten 4.1.1 while processing large datasets. I'm wondering if MongoKitten has memory leaks that might be the reason for my problems since XCode Instruments reports leaks related to MongoKitten (even though I'm not really able to interpret the output).

If I simply call try Database("mongodb://user:pass@host:port/dbname") (with correct credentials) I get the following output in Instruments:

All leaks in the list refer to either MongoKitten Server, Database, Collection or Document in their stack trace.

The single leaks might be very small, but if I process datasets of roughly 100 mio entries on the MongoKitten Server instance, they seem to sum up to many GB of RAM. Do you think that's possible?

Please let me know if you need any additional details. Thank you very much for your help!

[Joannis Orlandos]

Where do the leaks occur? I see IndexTrieNode inside a getBuildInfo. The build info document is stored inside the Database/Server. So that'll always be allocated. I'll definitely look into this since the many GBs mustn't occur, obviously. Do you have a concrete "process" that will generate memory leaks?

[danielpater]

Thanks for your fast reply. The leaks seem to occur during initialisation of Server, Socket, Database, Collection and Document classes. In the screenshot above in the stack trace, it occurred during Document.init, which in turn called "IndexTrieNode....".

To reproduce this, you only need to call try Database("mongodb://user:pass@host:port/dbname"). I did set up a minimal Package to test this.

To reproduce (Swift 4 / XCode 9):

1. Download https://github.com/danielpater/MongoKittenTest (4 loc)
2. Run swift package generate-xcodeproj
3. Open Xcode, add any valid mongo url in Sources > swifttest > main.swift
4. Run the "swifttest" target
5. In the project navigator sidebar, select the Debug Navigator tab > Memory > Profile in Instruments
6. Select "restart" in the dialogue window

Now the leaks should appear in the timeline. Click on the stop icon to see the stack traces. If you add code that performs any db queries, more leaks should appear during runtime.
Note that while true { } only keeps the program running for profiling and has nothing to do with the memory leaks.

[Joannis Orlandos]

I'm not finding many leaks. The few I am finding are the BuildInfo document and other data kept for communicating with MongoDB.

[Joannis Orlandos]

We did find leaks here and there in those releases. Those should be fixed. I'm closing this due to old age.