Pressure test cause a fatal error.

[Shun]

Describe the bug
A clear and concise description of what the bug is.

1. The memory is increased dramatically.

2. An crash error happened which point at the line below in MongoCore>Sessions.swift file.:

sessionManager?.releaseSession(serverSession)

And throw an error message below:

Fatal error: Can't form Range with upperBound < lowerBound: file

To Reproduce
Steps to reproduce the behavior.

I use Jmeter pressure tool to simulate 1000 users to do the infinite request on http://localhost:8080/v1/roles.

I use Vapor 4 to create routes and setup repositories, controllers, migrations,...

Here snip code is:

try v1.register(collection: RoleController())

try app.databases.use(.mongo(connectionString: dbURI), as: .mongo)

app.migrations.add(RoleMigration(), to: .mongo)

Expected behavior

1. The memory usage should be stable.
2. Do not crash.

Server-side Swift:

• OS: [macOS]
• Swift Version: 5.2
• MongoKitten Version [6.5.0] (fluent-mongo-driver from 1.0.0)

Additional context
I changed to fluent-postgres-driver package. Everything is going to quite well.

The memory was suppressed at the level nearly 23.4 MB. In additionally, the crash didn't happened.
In contrast, fluent-mongo-driver caused the memory over 200MB, and crashed unexpectedly.

[Joannis Orlandos]

Hey @qi-shun-wang,

I've got a few questions to help me investigate the issue, since it's occurring in quite a special place:

1. Are you using a transaction in that route?
2. Are you connected to a cluster?
3. Did you see what kinds of objects were lingering in memory?

Based on the large memory usage and the location of the crash, I'm assuming MongoKitten one of the following:

• MongoKitten is making a lot of connections, something that really mustn't occur beyond 1 connection per-server.
• A large amount of transactions happening don't happen on the same thread, and session storage is using memory for the large amount of sessions
• MongoKitten keeps disconnecting and reconnecting for some reason, and data lingers (probably sessions)

[Joannis Orlandos]

Also, are you able to submit a reproduction sample?

[Shun]

Are you using a transaction in that route? No. Just simple query on an route that is all I have done.
Are you connected to a cluster? No. An MongoDB instance run as container in the docker environment.
Did you see what kinds of objects were lingering in memory?

I push the current project as a reproduction sample.
Sample

You should slightly change these files before you reproduce the problem.

1. Sources/App/Setup/databases.swift

import Vapor
import FluentPostgresDriver
/// Register your application's databases here.
public func databases(_ app: Application) throws {
    // Configure a PostgreSQL database
    let dbURI =  Environment.get(AppEnvironment.MONGO_DB_CONNECTION_URI.value) ?? "mongodb://localhost:27017/test"
   try app.databases.use(.mongo(connectionString: dbURI), as: .mongo)
   //try app.databases.use(.postgres(url: "postgresql://optima:123456@localhost:5446/postgres"), as: .psql)
    
}
 

2. Sources/App/Setup/migrate.swift
   .mongo as migration target

app.migrations.add(PolicyMigration(), to: .mongo)

[Shun]

• d, and session storage is using memory for the large amount of sessions
• MongoKitten keeps disconnecting and reconnecting for some reason, and data lingers (probably sessions)

Is that creation of the session and destroy process improperly run on different thread than cause this problem unexpected ?

[Joannis Orlandos]

@qi-shun-wang after a long search without results I just noticed you're on MongoKitten 6.5.0 😅

There were a few important patches made since then:
https://github.com/OpenKitten/MongoKitten/releases/tag/6.5.1
https://github.com/OpenKitten/MongoKitten/releases/tag/6.5.2

[Shun]

Currently, fluent-mongo-driver depends on 6.5.0 version. I will patch the package and upgrade to 6.5.2 for testing the situation. If there is no problem happened, I will leave a message here. 👍

[Shun]

I pulled the fluent-mongo-driver and changed the version 6.6.3.
Let the sample's dependency refer to local fluent-mongo-driver in Package.swift.

.package(path: "../fluent-mongo-driver"),

Sadly, the problem still exist.
BTW, It happened nearly 2 mins when pressure testing had started.

[Joannis Orlandos]

I've found your issue! It was quite a bit simpler and different than what I was looking for. The implcit session was released to the session pool. So the session pool kept filling with references to this implicit (global) session ID. Let me know if 6.6.4 fixed your issue! And thanks for bringing this to my attention.

[Shun]

Brilliant! The problem have solved.