DIAGNijmegen / rse-pyswot

Python implementation of JetBrains/swot

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Use raw text files as Jetbrains did rather than in memory module to reduce memory overhead and startup time.

elonzh opened this issue · comments

I chose for runtime speed over startup time, memory usage is 5MB for a speedup of 37x. Versions prior to 0.2 used your suggested approach. Some profiling:

Our approach:

pyswot version: 0.2.1
Filename: test.py

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     6     20.1 MiB     20.1 MiB           1   @profile
     7                                         def fun():
     8     25.2 MiB      5.1 MiB           1     import pyswot


10000 loops for user@gmcc.tnua.edu.tw, best of 5: 0.00031309701799998944 s
10000 loops for user@nope.tnua.edu.tw, best of 5: 0.0003268956379999963 s

Raw text file approach:

pyswot version: 0.1.3
Filename: test.py

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     6     20.0 MiB     20.0 MiB           1   @profile
     7                                         def fun():
     8     20.0 MiB      0.0 MiB           1     import pyswot


10000 loops for user@gmcc.tnua.edu.tw, best of 5: 0.011768581818000012 s
10000 loops for user@nope.tnua.edu.tw, best of 5: 0.013157385768000003 s

test.py:

from memory_profiler import profile

from importlib.metadata import version
print(f"pyswot version: {version('pyswot')}")

@profile
def fun():
  import pyswot

fun()

import timeit
from statistics import mean

def time_profile(email):
    number = 10000
    timing = timeit.repeat(f"is_academic('{email}')", "from pyswot import is_academic")
    print(f"{number} loops for {email}, best of {len(timing)}: {mean(timing) / number} s")

time_profile("user@gmcc.tnua.edu.tw")
time_profile("user@nope.tnua.edu.tw")