Memory Leak when creating new connections at each API Request.

[Suvrat Jain]

Description

Making new connections at each request causes memory leak in python threading package threading.py at line no 908 and 913. Checked using tracemalloc for concurrent request.

Code to make connection and query

import tracemalloc
tracemalloc.start()
from fastapi import FastAPI, HTTPException, Query, APIRouter
import aiosqlite

app = FastAPI()

async def somefunction():
    db_path = "some.db"
    query = "select * from some_table limit 1"
    async with aiosqlite.connect(db_path) as db:
        cursor = await db.execute(query)
        row = await cursor.fetchone()
    if row is None:
        return None
    else:
        return row[0]
    
                

@app.post("/test"):
async def test():
    result = await somefunction()


@app.get("/get_memory_snapshot/")
async def get_memory_snapshot():
    try:
        snapshot = tracemalloc.take_snapshot()
        top_stats = snapshot.statistics("lineno")
        lines = []
        for stats in top_stats:
            lines.append(str(stats))
        return "\n".join(lines)
    except Exception as e:
        print("Error in getting memory allocation")

The issue was resolved when using global db connection.
async with aiosqlite.connect(db_path) as db: seems to be holding a threadlock even if the function context has ended.

run get_memory_snapshot after 1000 request to the the line consuming most memory

Details

• OS: Mac Ventura
• Python version: 3.8
• aiosqlite version: aiosqlite==0.19.0
• Can you repro on 'main' branch? yes
• Can you repro in a clean virtualenv? yes