Giters

memgraph / gqlalchemy

GQLAlchemy is a library developed with the purpose of assisting in writing and running queries on Memgraph. GQLAlchemy supports high-level connection to Memgraph as well as modular query builder.

Home Page:https://pypi.org/project/gqlalchemy/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Run concurrent transactions

bilalsp opened this issue · comments

commented

[New Feature Request]

Neo4j driver supports concurrent workflows to interact with the database in an asynchronous way as explained in documentation.

Documentation: Neo4j concurrent transactions

Apparently, gqlalchemy does not support concurrent workflows neither for Neo4j nor Memgraph.
I would appreciate if we implement this feature in gqlalchemy compatible with Python’s asyncio.

commented

Hi @bilalsp, thank you for opening the feature request :) We considered implementing this in the future, so I will leave this issue open to better prioritize later.

Is this a blocker for you? What are you working on?

commented

Hi @katarinasupe,
we are doing planing for our new async microservice which we will be implementing in FastAPI framework.
Graph database is really appealing for our usecase. So, we are considering all the aspects before making our decision to use graph database.

By the way, does Memgraph's python driver pymgclient support concurrent transactions similar to Neo4j driver?
I didn't find any resource related to this.

commented

@bilalsp Hi, I ran into this issue as well.
Here is what I found out so far:

  • The only way to parallellize queries with pymgclient is by multiprocessing - using ProcessPoolExecutor, or a helper library such as joblib. It works well, but you are limited by the number of cpu cores.
  • Multithreading doesn't really work with pymgclient because apparently it doesn't release the GIL for I/O activities.
  • The best solution I found is to use the neo4j driver v5 which has asyncio support. I tested it with 1000 concurrent connections and it was extremely fast. Unfortunately, neo4j v5 is incompatible with gqlalchemy that requires v4.4, so they can't exist in the same python env.