Potential performance issue: .to_json memory leak in pandas below 1.4 version
TendouArisu opened this issue · comments
Issue Description:
Hello.
I have discovered a performance degradation in the .to_json function of pandas version below 1.4. And I notice the repository depends on pandas 1.3.5 in Xwin-Coder/DS1000/ds1000_requirements.txt. I am not sure whether this performance problem in pandas will affect this repository. I found some discussions on pandas GitHub related to this issue, including #43877 and #45489.
I also found that Xwin-Coder/MT_bench/show_result.py used the influenced api. There may be more files using the influenced api.
Suggestion
I would recommend considering an upgrade to a different version of pandas >= 1.4 or exploring other solutions to optimize the performance of .to_json.
Any other workarounds or solutions would be greatly appreciated.
Thank you!